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FOREWORD 


The  present  series  of  conferences  was  initiated  on  l4  May  1954-  at  Frarik- 
ford  Arsenal  in  order  to  disseminate  information  on  the  methods  and  new  develop¬ 
ments  in  the  field  of  operations  research  to  a  large  number  of  government 
personnel.  On  20  May  1955  the  Diamond  Ordnance  Fuze  Laboratories  served  as 
host  to  the  second  conference,  and  the  third  meeting  was  held  24-25  May  1956 
at  Headquarters,  Ordnance  Weapons  Command.  Late  in  1958  Oscar  M.  Wells  ex¬ 
pressed  the  need  for  more  symposia  in  this  area;  and  thanks  to  the  invitation 
issued  by  Emory  L.  Atkins,  the  Fourth  Ordnance  Conference  on  Operations 
Research  was  held  at  Headquarters,  U.  S.  Ordnance  Missile  Command  on  1-3  April 
1959. 


Colonel  George  F.  Leist,  Ordnance  Corps  Commanding  Officer  of  the  Office 
of  Ordnance  Research,  served  as  chairman  of  the  first  General  Session  at 
which  Brigadier  General  John  G.  Shinkle,  Commander,  Army  Rocket  and  Guided 
Missile  Agency,  gave  the  Welcoming  Remarks.  He  also  presented  a  brief  his¬ 
tory  of  Redstone  Arsenal.  The  initial  invited  address,  by  Gayle  W.  McElrath 
of  the  University  of  Minnesota,  emphasized  the  need  for  careful  designing 
and  proper  planning  of  experiments.  The  second  invited  paper,  by  J.  E. 
Hoagbin,  The  Willow  Run  Laboratories  of  the  University  of  Michigan,  presented 
a  summation  of  the  development  of  a  mathematical  model  of  a  queuing  and 
scheduling  problem.  Dr.  John  W.  Coy,  White  Sands  Missile  Range,  served  as 
chairman  of  the  first  Technical  Session.  In  this  group  were  four  papers 
covering  such  topics  as  missile  system  testing,  war  games,  discounted  least 
squares,  and  waiting  line  theory.  Technical  Sessions  II  and  IV  had  Emory  L. 
Atkins  and  Robert  R.  Orr  from  the  Army  Ordnance  Missile  Command  as  their 
respective  chairmen.  Both  of  these  meetings  carried  security  classifica¬ 
tions  of  SECRET.  Mr.  Oscar  M.  Wells  presided  over  Technical  Session  III, 
in  which  four  papers  treated  demand  rates,  tolerance  limit  tests,  produc¬ 
tion  development,  and  applications  of  linear  programming.  The  final  phase 
of  the  conference  was  chairmaned  by  Ralph  Brown  of  Frankford  Arsenal.  In 
this  General  Session  0.  M.  Wells  discussed  various  types  of  arsenal  problems 
that  have  been  found  amenable  to  operations  research  techniques.  Lt.  Colonel 
Harold  W.  Rice,  Research  Section,  Plans  Division,  Deputy  Chief  of  Staff  for 
Logistics,  gave  an  invited  address  entitled  Army  Logistics  Research.  The 
closing  address  of  the  meeting  was  delivered  by  the  invited  speaker,  Herbert  P. 
Galliher ,  of  the  Massachusetts  Institute  of  Technology.  Dr.  Galliher  spoke  on 
Future  Operations  Research  Problems  in  Ordnance  Logistics. 

These  Proceedings  contain  thirteen  of  the  nineteen  papers  that  were  pre¬ 
sented  at  the  Fourth  Ordnance  Conference  on  Operations  Research.  An  appendix 
to  this  technical  manual  carrying  a  security  classification  of  SECRET  is 
being  issued.  This  appendix  will  contain  papers  by  J.  Duncan  Love,  Eugene  P. 
Visco,  John  P.  Young,  and  Richard  Zimmerman,  all  of  the  Operations  Research 
Office. 
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SOME  ASPECTS  OF  EXPERIMENTAL  DESIGN 
IN  OPERATIONS  RESEARCH 


Gayle  W.  McElrath 
Industrial  Engineering  Division 
Mechanical  Engineering  Department 
University  of  Minnesota 

INTRODUCTION,  Basically  the  ideas  of  designed  experiments  are  very 
fundamental  and  have  numerous  applications  in  the  various  areas  of  opera¬ 
tions  research.  It  becomes  apparent  that  the  usefulness  of  the  analysis 
of  data  depends  greatly  upon  a  systematic  planning  of  the  data  in  addition 
to  the  necessary  assumptions  of  independence  and  distribution.  The  intent 
of  this  paper  is  two-fold:  first,  to  acquaint  the  reader  with  concepts 
and  assumptions  ingredient  in  statistical  designs;  second,  to  present 
examples  which  illustrate  some  of  the  basic  properties  of  designed  exper¬ 
iments.  The  material  presented  is  not  intended  to  be  complete  in  nature. 

It  is  not  attempting  to  suggest  many  experimental  designs  together  with 
many  applications  nor  is  it  attempting  to  exhaust  all  the  possibilities 
of  designed  experiments.  It  must  follow,  then,  that  the  dominant  mission 
of  this  paper  is  to  establish  an  appreciation  and  dynamic  interest  strong 
enough  to  motivate  the  reader  to  further  study  designed  experiments  to¬ 
gether  with  the  appropriate  statistical  techniques  for  intelligent  analysis 
of  data  when  the  need  for  such  refinement  exists. 

In  operations  research  a  person  may  be  called  upon-  to  design  labora¬ 
tory  or  field  experiments  to  test  differences  between  alternate  methods, 
processes,  or  products.  He  may  also  analyze  operational  problems  by  taking 
samples  to  give  quick  estimates  of  short-run  processes  or  performance  fore¬ 
casts.  In  addition  he  may  be  called  on  to  spot  trouble  and  appraise  results. 
In  all  of  these  cases  one  is  faced  with  the  problem  of  decision  making  and 
estimation  in  the  face  of  random  variability. 

It  should  be  observed  that  the  field  of  statistical  decision  making 
and  estimation  is  relatively  new  but  one  that  is  gaining  support  at  a  rapid 
rate.  To  some,  the  pressure  of  work  and  what  seems  to  be  an  unreasonable 
amount  of  technical  terminology  combine  to  create  the  impression  that  too 
much  time  and  effort  are  necessary  to  understand  a  sufficient  amount  of 
the  design  and  analysis  of  experiments.  However,  a  sincere  effort  to  learn 
and  to  usd  these  techniques  is  early  rewarded  by  finding  that  the  basic 
fundamental  ideas  are  natural,  simple,  and  extremely  useful. 

GENERAL  CONSIDERATIONS  ,  A  classical  accepted  method  of  experimentation 
is  to  answer  a  single  question  about  a  single  variable.  There  is  an  attempt 
to  control  all  other  "factors"  except  the  one  of  interest.  Many  times  the 
planning  consists  of  trying  first  one  idea,  and  if  the  one  idea  does  not 
produce  "satisfactory  results"  then  try  another  idea.  However,  data  based 
on  this  type  of  experimentation  often  consists  of  a  number  of  disjoint  piece? 
of  information  that  cannot  be  easily  put  together.  It  is  possible  to  design 
experiments  such  that  all  combinations  of  the  different  factor  "levels"  can 
be  considered  in  order  that  their  "individual  effects"  together  with  their 
"interactions"  may  be  studied. 


2 


Fourth  Conference  on  Operations  Research 


When  the  term  DESIGN  OF  EXPERIMENT  is  used,  a  reference  is  made  to  a 
plan  of  choosing  a  sample.  For  fear  that  the  reader  think  only  in  terms  of 
sampling  a  process  that  exists ,  let  the  author  hasten  to  assure  him  that 
many  "designed  experiments"  are  called  upon  by  means  of  planned  sampling  to 
generate  information  about  a  process  that  does  not  actually  exist.  The. 
objective  is  to  discover  the  properties  of  the  process  if  the  process  did 
exist.  Much  of  the  exploratory  operations  research  is  carried  on  in  an 
environment  where  the  process  does  not  exist.  Of  course,  a  second  situation 
exists  when  there  is  actually  a  real  process  and  the  objective  is  to  deter¬ 
mine  certain  measurable  characteristics  relative  to  this  process.  Again 
there  are  special  "designed  experiments"  which  can  give  more  complete  and 
precise  information  about  this  process  than  just  haphazard  sampling.  The 
analysis  of  the  data  in  the  designed  experiment  forces  one  to  use  the  con¬ 
cepts  of  statistical  significance  and  estimation  which  in  turn  force  one  to 
think  in  terms  of  decisions  and  estimations  with  calculated  risks. 

Again  it  should  be  emphasized  that  statistical  decision  making  and 
estimation  via  designed  experiments  is  done  in  the  face  of  random  varia¬ 
bility.  This  philosophy  is  fundamental  to  many  areas  of  operations 
research. 

Anyone  can  try  to  make  sense  out  of  a  set  of  observational  or  experi¬ 
mental  data.  However,  the  ability  to  scrutinize,  to  look  beneath  the  surface 
and  to  discern  relationships  from  given  data  is  aided  by  scientific  methods 
of  collecting,  analyzing,  and  interpreting  data.  The. modem  methods  of 
designed  experiment  and  analysis  deal  with  such  questions  as : 

1.  What  factors  are  considered  important  and  of  interest? 

2.  What  type  of  designed  experiment  should  be  used? 

3.  How  shall  a  program  of  obtaining  the  data  be  planned? 

4.  How  shall  the  data  be  analyzed? 

5.  What  conclusions  are  we  entitled  to  draw  from  the  data? 

6.  How  shall  the  experimental  error  be  estimated  and  how  reliable 
are  the  conclusions? 

To  answer  these  questions  completely  in  research  engineering,  or 
production,  is  an  impossibility.  However,  we  may  gain  insight  of  the 
basic  philosophy. 

The  first  three  questions  in  the  above  list  are  often  slighted  or  are 
not  considered  before  the  data  is  gathered.  Too  many  times  a  project  is 
started,  data  is  obtained,  and  a  post-mortem  analysis  of  the  data  is  held 
in  a  faint  hope  of  finding  something  "significant."  The  criterion  of  sig¬ 
nificance  in  this  situation  often  depends  upon  personal  judgment.  Before 
an  effective  program1  of  obtaining  data  can  be  planned  a  STATEMENT  OF  THE 
PROBLEM  is  indeed  necessary.  This  statement  might  very  well  take  the  form 
of  a  list  of  questions  concerning  the  factors  of  interest.  When  this  list 
of  questions  has  been  generated,  perhaps  after  sufficient  exploration,  a 
most  important  prerequisite  of  a  designed  experiment  has  been  satisfied. 
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The  problems  associated  with  experimental  operations  research  are 
basically  problems  that  demand  an  analysis  of  the  variation  of  the  data 
in  an  attempt  (a)  to  determine  which  variables  are  significant  and  (b) 
to  determine  interval  estimates  of  the  effects  of  the  variables  considered. 
To  this  end,  statistical  methods  may  be  used  to  analyze  the  data  from  the 
designed  experiment  to  calculate  the  total  amount  of  variation  which  is 
sub-divided  into  categories  traceable  to  definite  sources •  The  relative 
sizes  of  these  key  categories  of  variation  are  compared  with  the  size  of 
the  “'experimental  error. w:  The  significance  of  the  effects  of  the  variables 
together  with  the  interval  estimates  can  be  judged  as  an  outcome  of  these 
comparisons.  The  sensitivity  of  the  experiment  depends  upon  the  magnitude 
of  this  experimental  error j  this  error  becomes  the  yardstick  of  the  exper¬ 
iment.  The  size  of  the  error  determines  the  ability  to  pick  up  effects 
that  are  significant  and  thereby  dictates  a  “level  of  comparison.*  A  large 
experimental  error  might  veiy  well  mask  the  effects  of  important  variables 
which  are  present.  The  ideal  situation  would  dictate  a  miniwnim  experimental 
error;  however,  we  are  usually  forced  to  indicate  an  “economic"  minimum. 

Our  problem  in  planning  a  program  for  obtaining  data  is  two-fold*  namely, 
that  of  estimating  the  experimental  error,  and  that  of  determining  an 
economic  experimental  error  (yardstick).  A  pre-planned  experiment  provides 
at  least  three  ways  of  controlling  the  experimental  error.  They  are  as 
follows s 

1.  The  degree  of  refinement  of  the  measurement  techniques. 

2.  The  selection  of  the  number  of  replications. 

3.  The  choice  of  the  DESIGN  OF  THE  EXPERIMENT, 

SOME  BASIC  DESIGNS.  The  SUBJECT  of  a  designed  experiment  is  tine 
dependent  measureable  characteristic.  Other  variables  which  appear  to 
affect  the  subject  are  called  FACTORS.  The  factors  of  an  experiment  may 
be  varied  deliberately,  from  trial  to  trial.  The  relation  of  subject  to 
factors  can  be  measured  by  proper  planning  of  the  given  factors  controlled 
at  a  given  number  of  LEVELS.  A  TREATMENT  is  that  combination  of  levels  of 
all  of  the  factors  which  are  considered  in  a  given  trial.  The  numerical 
value  corresponding  to  a  given  trial  is  the  OBSERVATION  or  RESPONSE. 

Consider  the  following  introductory  remarks  that  suggest  the  evolution 
of  a  designed  experiment  which  allows  an  unbiased  estimate  of  the  experi¬ 
mental  error  a  nd  at  the  same  time  introduces  a  method  to  minimize  the  error 
by  restricting  the  randomization. 

The  following  problem  is  used  to  present  some  very  important  funda¬ 
mentals  which  should  be  a  part  of  operations  research  planning  of  experi¬ 
ments.  The  problem  is  to  test  for  a  differential  effect  of  materials 
from  four  vendors.  The  material  is  aluminum  clad  and  is  to  be  spot 
welded.  The  experiment  is  motivated  by  an  interest  to  select  the  material 
which  responds  in  the  best  manner  to  spot  welding  at  given  specifications. 
However,  it  is  believed  that  specific  types  of  spot  welders  have  different 
characteristics  and  that  these  characteristics  become  important  when  the 
engineer  is  faced  with  a  recommendation  for  routing  the  material  in  a 
given  process.  Consider  the  subject  to  be  tensile  strength  and  the  factor 
to  be  materials.  We  shall  label  the  materials  as  A,  B,  C,  and  D.  Let  us 
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consider  a  very  practical  situation  in  which  the  aluminum  clad  material 
is  welded  by  four  different  machines,  1,  2,  3,  and  U.  Although  the 
machines  are  set  to  spot  weld  to  specification,  it  will  be  necessary  to 
consider  these  machines  as  another  factor. 

It  is  obviously  faulty  experimental  design  to  lay  out  the  experiment 
as  shown  schematically  in  Figure  1.  If  a  difference  appears  we  do  not 
know  whether  the  difference  is  due  to  machines  or  materials.  The  materials 
effect  is  completely  "confounded"  with  the  machines  effect  and  the  two  are 
not  separable. 


Machine  1 

Material  A 
Material  A 
Material  A 
Material  A 


Machine  2 

Material  B 
Material  B 
Material  B 
Material  B 


Machine  3 

Material  C 
Material  C 
Material  G 
Material  C 


Machine  I4. 

Material  D 
Material  D 
Material  D 
Material  D 


Figure  1, 


A  suggested  improvement  on  the  above  procedure  is  to  "completely 
randomize"  the  materials  among  the  machines.  For  example,  we  could  iden¬ 
tify  A,  B,  C,  D  four  times  by  16  random  numbers.  Each  machine  would  spot 
weld  the  four  materials  as  they  occurred  by  this  "complete  randomization." 
See  Figure  2. 


Machine  1 

Machine  2 

Machine  3 

Machine  ij. 

Material  A 

Material  B 

Material  C 

Material  B 

Material  C 

Material  A 

Material  D 

Material  B 

Material  C 

Material  D 

Material  B 

Material  A 

Material  A 

Material  C 

Material  D 

Material  D 

Figure  2. 

The  design 

suggested  in  Figure 

2  is  statistically  sound.  We  can  take 

the  averages  of  the  four  results  for  each  material  to  estimate  their  pro¬ 
cess  means.  However,  in  a  completely  randomized  design  the  experimental 
error  may  be  inflated  because  it  includes  the  differences  of  the  machines. 
Thus  the  experiment  might  not  be  as  sensitive  as  possible. 

Another  suggestion,  in  the  face  of  the  above  discussion,  is  to  have 
each  machine  process  each  material  once  as  indicated  in  Figure  3. 


Machine  1 

Material  A 
Material  B 
Material  C 
Material  D 


Machine  2 

Material  A 
Material  B 
Material  C 
Material  D 


Machine  3 

Material  A 
Material  B 
Material  C 
Material  D 


Machine  k 

Material  A 
Material  B 
Material  C 
Material  D 


Figure  3 
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In  this  situation  the  differences  in  materials  are  completely 
independent  of  machines  .  However,  in  order  not  to  introduce  any  possible 
bias  into  the  experiment  because  of  the  identical  ordering  for  each  machine, 
a  further  refinement  is  suggested.  That  is,  to  randomize  the  order  of  ma¬ 
terials  to  be  spot  welded  for  each  machine  as  in  Figure  h* 


Machine  1 

Material  A 
Material  C 
Material  D 
Material  B 


Machine  2 

Material  C 
Material  B 
Material  D 
Material  A 


Machine  3 

Material  B 
Material  C 
Material  D 
Material  A 


Machine  U 

Material  D 
Material  B 
Material  A 
Material  D 


Figure  lu 

The  above  basic  design  is  one  of  the  simplest  and  perhaps  most  fre¬ 
quently  used  experimental  designs  used  in  operations  research,  and  is  called 
the  "randomized  block"  design.  A  conventional  notation  would  label  machines 
as  "blocks"  and  materials  as  "treatments."  The  blocks  are  so  chosen  that 
units  in  a  block  have  some  common  feature  or  background  likely  to  them 
uniform.  If  possible,  every  unit  in  a  block  receives  identical  handling 
throughout  the  experiment  except  for  one  factor:  the  experimental  treat¬ 
ment  (materials,  in  this  example).  Thus  we  identify  this  experiment  as 
single  grouping  to  reduce  the  experimental  error.  The  statistical  tech¬ 
nique  applicable  to  analyze  the  data  obtained  from  the  randomized  block 
design  is  the  "analysis  of  variance."  By  using  the  analysis  of  variance 
we  can  divide  the  total  variability  into  three  "sources  of  variation" ; 
namely,  materials,  machines,  and  experimental  error.  Thus,  material,  effect 
can  be  compared  with  the  experimental  error  to  determine  the  materials 
significance,  independent  of  whether  there  is  a  significant  variation  from 
machine  to  machine. 


Note  that  the  possibility  for  gain  in  sensitivity  in  a  randomized 
block  design  arises  from  further  restricting  the  randomization  which  restats 
in  the  grouping  that  is  employed  in  the  experiment.  The  principle  is  to  use 
groupings  within  which  the  product  is  likely  to  be  less  variable  the 
product  as  a  whole.  This  is  an  analog  to  rational  subgroups  which  is  used 
so  frequently  in  analysis  of  process  variation  by  statistical  quality  con¬ 
trol  techniques. 


Features  of  randomized  blocks:  * 


1.  One  category  of  factors  can  be  permitted  to  vary  from  one  block 
to  another,  yet  the  main  component  of  this  variation  becomes 
excluded  from  the  residual  error  of  the  experiment. 

2o  Any  number  of  treatments  that  is  desired  can  be  used,  so  long  as 
each  block  has  the  same  set  of  treatments. 


*  See  Quality  Control  Handbook,  J.  M.  Juran,  McGraw-Hill  Book  Co.,  Inc., 
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3.  As  many  repeat  tests  (called  "replications”)  as  are  wanted  can 
be  obtained  by  adding  blocks. 

ll«  Analysis  of  remaining  data  can  still  be  made  even  if  some  failure 
in  process  or  test  eliminates  an  entire  block. 

MACHINES 

1  2  3  k 

I  Material  B  Material  A  Material  C  Material  D 

II  Material  D  Material  C  Material  A  Material  B 

III  Material  A  Material  D  Material  B  Material  C 

IV  Material  C  Material  B  Material  D  Material  A 

Figure  !>• 

A  natural  extension  of  the  concept  of  single  grouping  to  reduce  errors 
is  to  suggest  a  double  grouping.  This  is  possible  by  an  additional  res¬ 
triction  of  the  randomization  of  units  to  be  tested. 

For  example,  it  is  well  known  that  the  tensile  strength  of  the  spot 
welds  can  be  influenced  by  the  techniques  of  the  operators  I,  II,  III,  and 
IV,  on  the  machines.  By  advanced  planning  the  design  with  double  grouping 
requires  that  each  material  is  processed  just  once  respectively  with  each 
machine  and  operator  combination*  It  follows,  then,  that  the  number  of 
machines,  materials  and  operators  must  all  be  the  same.  An  experimental 
design  of  this  type  is  called  a  Latin  -  square  experiment.  The  scheme  is 
given  in  Figure  f>. 

As  before,  the  observed  variation  in  tensile  strength  can  be  classi¬ 
fied.  If  the  subject  of  the  experiment  satisfies  certain  basic  assumptions 
to  the  analysis  of  variance,  we  can  measure  the  significance  of  the  three 
sources  of  variation;  namely,  the  materials,  the  machines,  and  the  operators 
The  criterion  of  significance  again  is  determined  by  comparison  with  the 
experimental  error. 

However,  there  exists  a  restricting  assumption  in  the  analysis  of  the 
Latin  -  square  data.  That  is,  this  design  will  not  measure  "interaction." 
Interaction  is  basically  an  additional  response  in  one  factor  due  to  the 
presence  of  another  factor  at  a  specific  level.  The  assumption  of  no  inter¬ 
action  can  either  be  tacit  or  explicitly  determined.  (Much  of  our  classical 
experimentation  incorrectly  assumes  that  the  factors  are  independent  and 
that  the  interaction  does  not  exist.) 

An  elementary  experiment  which  has  the  interaction  effect  designed 
into  it  is  a  two  factor  design  with  repeated  measurements.  If  there  is  a 
possibility  for  significant  interaction,  the  estimate  of  the  experimental 
'error  for  use  in  investigating  significant  effects  of  the  factors  is  ob— 
tained  from  measurements  repeated  under  the  same  conditions.  This  repli¬ 
cation  will  allow  us  to  analyze  the  data  more  fully. 
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Suppose  that  in  a  study  of  tensile  strength,  the  two  factors  are 
materials  and  machines,  and  that  a  sample  of  three  coupons  of  the 
aluminum  clad  material  are  spot  welded  by  the  same  machine.  The  design 
may  take  the  appearance  of  Figure  6. 


1 

Sample  1A 
Sample  1A 
Sample  1A 

MACHINES 

2 

Sample  2A 
Sample  2A 
Sample  2A 

3 

Sample  3A 
Sample  3A 
Sample  3A 

Sample  IB 
Sample  IB 
Sample  IB 

Sample  2B 
Sample  2B 
Sample  2B 

Sample  3B 
Sample  3B 
Sample  3B 

Sample  1C 
Sample  1C 
Sample  1C 

Sample  2C 
Sample  2C 
Sample  2C 

Sample  3C 
Sample  3C 
Sample  3C 

Sample  ID 
Sample  ID 
Sample  ID 

Sample  2D 
Sample  2D 
Sample  2D 

Sample  3D 
Sample  3D 
Sample  3D 

Figure  6. 

To  analyze  the  data  of  this  experiment  we  can  again  use  the  statistical 
technique  called  the  analysis  of  variance  »  We  divide  the  source  of  total 
variation  into  four  different  categories*  namely,  materials,  machines, 
interaction  between  materials  and  machines,  and  experimental  error.  Thus 
we  have  a  yardstick  in  terms  of  the  experimental  error  to  determine  the  sig¬ 
nificance  of  the  factors  involved.  At  least  by  this  designed  experiment, 
together  with  its  analysis,  we  can  determine  statistically  significant  fac¬ 
tors  in  an  objective  manner  and  can  determine  the  risks  of  wrong  decisions. 
The  wrong  decisions  are  of  two  types;  namely,  the  decision  that  there  is 
no  significant  difference  when  the  difference  actually  exists  or  the  deci¬ 
sion  that  there  is  a  significant  difference  when  the  difference  does  not 
exist. 

SUMMARY.  Previously  we  asked  four  questions  concerning  scientific 
methods  of  handling  data.  The  emphasis  of  this  paper  was  concerned  with 
basic  concepts  ingredient  in  the  design  of  the  experiments. 

To  repeat,  the  classical  accepted  method  of  experimentation  has  been 
to  answer  a  single  question  about  a  single  variable.  That  is,  there  was 
an  attempt  to  control  all  other  factors  except  the  one  of  interest.  How¬ 
ever,  it  is  often  found  that  data  based  on  the  control  of  all  factors, 
except  the  one  under  investigation,  consists  of  a  number  of  disjoint  pieces 
of  information  that  cannot  be  easily  put  together. 
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By  method  of  designed  experiments,  plans  are  available  which  consider 
the  investigation  of  the  effects  of  a  number  of  independent  factors  at  the 
same  time.  The  author  wishes  to  emphasize  that  the  desired  information  be 
planned  and  designed  into  the  experiment  before  the  actual  data  is  obtained. 

There  are  several  advantages  to  be  gained  by  the  operations  researcher 
who  thinks  in  terms  of  designed  experiments.  Three  important  advantages 
may  be  summarized  as  follows! 

1.  Greater  efficiency  -  therefore,  greater  economy  of 
experimentation . 

2.  Results  can  be  obtained  which  apply  over  a  greater 
range  of  conditions. 

3.  Main  effects  together  with  their  interactions  can 
be  estimated. 

The  inferences  drawn  from  designed  experiments  are  valid  inferences 
containing  a  measure  of  uncertainty.  However,  these  inferences  are  rigorous 
when  made  by  statistical  analysis,  since  they  include  a  probability  measure 
of  the  amount  of  uncertainty  involved.  This  achievement  makes  it  possible 
to  determine  the  reliability  of  the  process  of  acquiring  new  knowledge  by 
observation.  Thus  it  becomes  possible  to  accomplish  one  important  goal  of 
operations  research. 
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MATHEMATICAL  MODELS  AND  THE  UTILIZATION  OF  THESE  MODELS 


J.  E.  Hoagbin 

Operations  Research  Department 
Willow  Run  Laboratories 
The  University  of  Michigan 

It  is  frequently  necessary  to  conduct  experiments  using  a  mathematical 
model  of  a  system  rather  than  the  system  itself.  Some  of  the  possible  reasons 
for  this  are: 

lo  It  is  necessary  to  experiment  with  a  system  that  has  not 
yet  been  built,  although  intended  properties  of  components 
are  known. 

2.  It  is  impossible  to  assemble  all  components  for  experimenta¬ 
tion,  even  though  they  have  been  built. 

3«  It  is  impossible  or  uneconomical  to  train  operators  in  the 
use  of  many  alternative  rules  of  operation.  The  plan  is 
to  experiment  with  many  rules  of  operation  and  then  train 
operators  to  use  the  rules  that  have  been  found  to  be 
best. 

it.  Because  of  the  many  stochastic  elements  in  the  systems,  it 
is  necessary  to  conduct  many  experiments  to  estimate  the 
mean  and  the  variance  of  the  property  that  is  to  be  measured 
during  the  experimentation  program.  It  is  economically 
unsound  to  conduct  many  experiments  with  the  system,  or  the 
system  cannot  he  made  available  for  long  periods  of  exper¬ 
imentation. 

It  is  impossible  to  conduct  the  experiments  for  moral  rea¬ 
sons.  For  example,  we  cannot  use  real  bullets  in  a  maneuver 
or  air  exercise. 

Frequently  equations  can  be  written  describing  system  behavior.  The 
equations  represent  a  model  of  the  system  and  experimentation  with  the  model 
consists  of  solving  the  equations  for  different  values  of  parameters.  Such 
mathematical  models  are  well  known  and  include  the  models  that  have  grown 
out  of  linear  programming,  for  example. 


The  rapid  development  of  Monte  Carlo  techniques  and  the  large  scale 
digital  computer  have  made  possible  the  construction  of  a  whole  new  class 

of  model.  It  is  this  latter  class  of  model  that  will  be  discussed  in  this 
talk. 


DIGITAL  COMPUTER  MODELS. 

Digital  computer  models  can  now  be  constructed  to  study  problems  that 
are  difficult  or  impossible  to  study  by  the  use  of  classical  mathematical 
analysis  for  such  reasons  ass 
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1.  Equations  expressing  system  behavior  cannot  be  solved  because 
of  discontinuities  and  non-linearities. 

2.  Knowledge  of  the  transient  behavior  of  the  system  is  necessary, 
and  existing  methods  of  classical  analysis  give  no  information 
about  transient  behavior. 

3.  There  are  stochastic  elements  that  cannot  be  handled  by 
classical  methods. 

Many  military  systems  involve  the  flow  of  information  or  material;  and 
attention  is  drawn  to  capacities,  accuracies,  and  time  delays.  Simulation 
of  such  a  system  requires  the  solution  of  many  technical  computations  as 
well  as  the  solution  of  a  complex  waiting-line  problem. 

A  SIMPLE  DEMONSTRATION  OF  THE  TTSE  OF  MONTE  CARLO  TO  STUDY  A  WaiTING-LINE  PRO- 
BlEtl.  •  — - 


Consider  a  hypothetical  operator  who  must  process  inputs.  As  new  inputs 
arrive,  the  operator  takes  them  in  turn  if  he  is  free.  If  he  is  busy  when 
an  input  arrives,  the  input  must  wait  for  service  in  a  queue  or  waiting  line. 
Occasionally  the  operator  makes  mistakes  and  produces  scrap,  rather  than 
properly  finished  inputs. 

The  total  time  required  for  inputs  to  pass  through  this  simple  system, 
the  time  spent  by  inputs  waiting  for  service,  and  the  number  of  inputs  that 
are  properly  processed  in  a  given  period  depend  on  the  following:  (1)  the 
distribution  of  arrivals  or  randomness  of  the  arrivals  and  average  rate  of 
arrival,  (2)  the  distribution  of  operator  holding  times  or  randomness  of 
the  time  required  for  the  operator  to  process  an  input  and  the  average  rate 
at  which  the  operator  can  process  inputs,  and  (3)  the  probability  the  opera¬ 
tor  will  make  a  mistake  and  produce  scrap  rather  than  properly  finished  in¬ 
puts. 


The  distribution  of  arrivals  may  be  a  function  of  parameters,  but  assume 
the  distribution  is  that  shown  in  Table  I.  Presentation  of  the  histogram 
used  to  produce  Table  I  in  tabular  form  will  be  useful  in  the  demonstration 
of  sampling  from  distributions  that  will  come  later. 

It  is  important  to  emphasize  that  the  long  and  short  intervals  between 
inputs  occur  randomly  and  are  not  attributable  to  any  known  factors.  Thus, 
just  as  any  two-digit  random  number  between  00  and  99  is  equally  likely  to 
occur  on  a  singly  drawing,  any  of  the  time  intervals  can  occur,  although 
the  occurrence  of  some  are  more  probable  than  others. 

Just  as  there  is  some  randomness  in  the  intervals  between  successive 
arrivals,  there  is  randomness  in  the  time  required  for  the  operator  to  pro¬ 
cess  an  input.  When  factors  affecting  the  length  of  service  times  or  hold¬ 
ing  times  are  known,  the  distribution  of  holding  times  can  be  made  a  function 
of  these  factors. 
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Table  II  shows  the  distribution  of  service  times  for  the  operator  under 
consideration,,  Note  that  the  distribution  is  a  function  of  the  length  of 
the  queue* 

TABLE  I,  ASSUMED  DISTRIBUTION  OF  ARRIVALS  OF  INPUTS  TO  A  HYPOTHETICAL  OPERATOR* 

,To,^fe  the  table,  draw  a  two-digit  random  number,  find  the  row  correspond- 
ing  to  this  number  in  the  first  column,  and  read  a  time  interval  in  the  column 
to  the  right* 


Random 

Number 

Time 

Interval 

01-02 

3«0  minutes 

03-05 

2.5 

06-07 

2.0 

08-11 

1.75 

12-1^ 

1.5 

16-22 

1.25 

23-2  9 

1.0 

30-ljl 

.75 

1*2-1$ 

.5 

5d‘-57 

.1* 

58-69 

.3 

70-79 

*2 

80-87 

ao 

88-91* 

.05 

95-100* 

.03 

*Read  00  as  100* 


TABLE  I 


1  k 
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TATtr.R  u,  HYPOTHETICAL  DISTRIBUTION  OF  HOLDING  TIMES  FOR  AN  OPERATOR  WHOSE 
HOLDING  TIMES  ARE  A  FUNCTION  OF  THE  LENGTH  OF  THE  QUEUE.  ~~~~~~~~ 

To  use  this  table:  draw  a  two-digit  random  number,  look  in  the  column 
corresponding  to  the  length  of  the  queue,  and  read  the  length  of  time  re¬ 
quired  for  the  operator  to  perform  his  function  in  the  column  on  the  right, 
opposite  the  random  number  chosen. 


Length  of  Queue 

Time  Required 

0“ 

I 

2 

3  or  more 

00-01 

.3 < minutes 

00-01 

02-10 

•U 

00-01 

02-17 

11-50 

.5 

00-01 

02-10 

18-145 

51-80 

.6 

02-05 

11-30 

l|6-75 

81-95 

.7 

06-20 

31-55 

76-90 

96-98 

.8 

21-U; 

56-78 

91-96 

99 

.9 

la-60 

79-88 

97-99 

1.0 

61-80 

89-96 

1.1 

81-90 

97-98 

1.2 

91-95 

99 

1.3 

96-98 

1.U 

99 

1.5 

1.6 

1.7 

TABLE  II 
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TABLE  III,  HYPOTHETICAL  PROBABILITY  AN  OPERATOR  WILL  PRODUCE  SCRAP. 
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Length  of 
Queue 


Probability 
of  Producing  — 
Scrap 


0  .10 

1  .15 

2  .18 

3  or  more  .20 


TABLE  III 

The  probability  an  operator  ■will  produce  scrap  may  also  be  a  function 
of  parameters.  Table  III  shows  the  probability  to  be  used  in  the  demonstra¬ 
tion  and  shows  this  probability  is  also  a  function  of  the  length  of  the  queue. 
Thus,  when  the  operator  works  faster,  he  makes  more  mistakes. 

To  begin  the  demonstration,  assume  the  operator  has  no  inputs  waiting, 
although  the  simulation  could  begin  with  some  waiting  line  of  interest.  It 
is  possible  to  determine  in  advance  when  inputs  will  arrive.  The  times  of 
arrival  of  five  inputs  will  be  determined.  Draw  five  two-digit  random  numbers 
and  get  85,  32,  38,  61*,  and  97.  Consulting  Table  I,  the  first  input  will 
arrive  at  time  .1  minutes,  the  second  .75  minutes  thereafter,  and  so  on. 

Call  events  of  type  El  the  event  of  the  arrival  of  an  input.  We  can  construct 
the  following  table; 

El 

.10 

.85 

1.6 

1.9 

1.93 

The  operator  is  free  when  the  input  arrives  at  time  0.1.  To  determine 
when  he  will  finish  working  on  the  input,  draw  a  random  two-digit  number 
and  get  1 1*.  Consulting  Table  II,  the  operator  will,  require  .8  minutes  to 
process  the  input.  Thus,  the  operator  will  finish  at  time  .9  which  is  .8 
minutes  after  he  started. 
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To  determine  whether  the  operator  will  produce  scrap:  Consulting  Table 
III,  if  the  number  drawn  is  01-10,  the  operator  produced  scrap.  Draw  the 
number  79 .  The  operator  will  produce  a  properly  finished  input. 

Call  the  event  of  type  E2  the  event  that  the  operator  has  just  finished 
processing  an  input.  We  now  have  the  following  table  from  which  the  first 
EL  was  deleted: 

EL  E2 

.85  .9 

1,6 
1.9 
1.93 

And  just  prior  to  time  .85,  the  status  of  the  system  can  be  summarized 
as  follows : 

Inputs  in  the  queue:  None 

Operator:  Processing  a  properly  finished  input 
Number  of  properly  finished  inputs:  None 
Number  of  scrapped  inputs :  None 

Proceed  to  the  next  event  that  will  occur.  This  is  the  event  of  type 
EL  at  time  .85  when  a  new  input  will  arrive.  Since  the  operator  is  still 
busy  at  this  time,  the  new  input  must  go  into  the  queue;  and  the  status  of 
the  system  can  be  summarized  as  follows  just  before  time  ,9: 

Inputs  in  the  queue:  1 

Operator:  Busy  processing  a  good  input 

Number  of  properly  finished  inputs:  None 

Number  of  scrapped  inputs:  None 

The  events  that  will  occur: 

EL  E2 

i.6  .9 

1.9 
1.93 
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Proceed  immediately  to  time  09  to  consider  what  happens  when  the  opera¬ 
tor  finishes  processing  the  input.  At  this  time  add  1  to  the  list  of  properly 
completed  inputs,  delete  one  from  the  queue,  and  determine  when  the  operator 
will  finish  processing  the  second  inpute 

Draw  the  random  number  98  and  consult  Table  II  to  learn  the  operator 
will  require  l.U  minutes  to  process  this  inpute  Also  draw  the  random  number 
08  and  learn  the  input  is  destined  to  become  scrap.  Since  the  new  E2  will 
come  at  time  2.3,  the  times  recorded  now  ares 


El 

E2 

1.6 

2.3 

1.9 

1«93 

Thus,  just  prior  to  time  1,6,  the  status  of  the  system  iss 
Inputs  in  the  queues  None 
Operators  Producing  a  scrap  input 
Number  of  properly  finished  inputs s  1 
Number  of  scrapped  inputs?  None 

If  we  proceed  to  the  next  event  that  will  occur,  we  find  the  operator 
busy  when  the  input  arrives  at  time  1,6$  hence,  we  add  1  to  the  queue.  The  next 
two  events  are  also  associated  with  arrivals  of  inputs;  and  just  prior  to 
time  2.3,  there  will  be  three  inputs  in  the  queue.  At  time  2,3,  add  1  to 
to  the  list  of  scrapped  inputs,  subtract  one  from  the  queue,  and  again  de¬ 
termine  when  the  operator  will  finish  processing  the  next  input. 

When  the  fifth  input  has  been  processed,  no  more  times  will  remain  in 
the  list  of  events;  and  the  simulation  of  the  operator  processing  five  in¬ 
puts  will  be  over. 

The  emphasis  in  the  demonstration  was  on  sampling  from  distributions 
and  tracing  of  inputs.  Nothing  was  said  about  recording  information  con¬ 
cerning,  for  example,  the  total  time  required  to  get  inputs  through  the 
whole  system.  However,  it  is  evident  that  one  can  record  the  time  of  arrival 
of  inputs,  the  time  service  begins  on  each  input,  and  the  time  processing 
of  each  input  was  complete.  From  this  information  it  is  possible  to  compute 
the  average  waiting  time  and  the  average  time  required  for  the  system  to  pro¬ 
cess  inputs,  including  waiting  times.  Also,  it  is  possible  to  record  indi¬ 
vidual  waiting  times  and  group  these  as  an  estimate  of  the  distribution  of 
waiting  times. 
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Because  of  the  randomness  of  the  arrivals  and  holding  times,  it  is  evident 
that  a  single  tracing  of  five  inputs  through  the  system  is  not  sufficient 
to  determine,  for  example,  the  average  time  required  for  five  inputs  to  pass 
through  the  system.  Many  replications  of  the  experiments  would  be  required 
to  estimate  this  mean  time  and  the  distribution  of  times  required  for  the 
system  to  process  five  inputs.  After  a  few  trials,  it  is  possible  to  esti¬ 
mate  the  mean  and  the  variance  and  arrive  at  a  decision  about  how  many  more 
replications  are  necessary  in  order  to  estimate  the  mean  or  the  whole  dis¬ 
tribution  to  the  desired  accuracy. 

The  preceding  demonstration  not  only  showed  how  Monte  Carlo  procedures 
can  be.  used  in  system  simulation  but  also  indicated  the  types  of  computations 
performed  by  digital  computers  used  in  some  system  simulations.  Because 
of  the  way  components  were  represented,  it  was  possible  to  proceed  in  trac¬ 
ing  inputs  by  a  purely  computational  procedure.  It  is  possible  to  draw  flow 
charts  that  give  step-by-step  instructions  that  can  be  programmed  for  a  large 
digital  computer.  Figure  1  shows  the  master  flow  chart  used  in  the  demon¬ 
stration.  Figure  2  and  Figure  3  show  the  flow  charts  containing  the  instruc¬ 
tions  that  were  followed  when  either  of  the  two  event  types  was  the  next 
occurring  event  in  the  event  list  or  time  status  record. 

Flow  charts  such  as  those  contained  in  Figures  2  and  3  can  be  drawn 
by  people  familiar  with  a  particular  system  and  the  reason  for  the  system 
simulation.  Computer  programmers  can  translate  these  flow  charts  into  com¬ 
puter  language.  Considerable  ingenuity  is  required  on  the  part  of  the  com¬ 
puter  programmer  to  find  ways  to  put  into  the  available  storage  all  of  the 
instructions  and  component  properties  necessary  for  the  simulation  of  a  com¬ 
plex  system.  Considerable  work  has  been  done  on  generation  of  random  numbers 
in  computers  and  on  sampling  from  distributions.  No  effort  will  be  made  here 
to  discuss  these  elegant  techniques.  The  techniques  chosen  for  the  demonstra¬ 
tion  could  be  used  in  a  digital  computer  simulation  but  are  not  elegant. 

The  purpose  of  the  preceding  paragraphs  was  to  give  a  demonstration  of 
how  one  might  go  about  setting  up  and  operating  a  computer  simulation  of  a 
simple  system.  The  following  steps  are  necessary  in  such  an  efforts 

1.  Become  familiar  with  the  system,  its  components,  and  the 
rules  of  operation. 

2.  Determine  what  questions  are  to  be  answered  by  the  simu¬ 
lation. 

3.  Determine  what  data  are  to  be  collected  during  the  simu¬ 
lation.  That  is,  determine  what  information  must  be 
printed  out  by  the  computer  in  order  to  answer  the  ques¬ 
tions  that  are  to  be  answered. 

U.  Decide  on  the  representation  of  components.  For  example, 
in  the  demonstration  we  represented  the  operator  and  his 
equipment  by  a  distribution  of  holding  times  and  a  prob¬ 
ability  of  making  scrap* 


is 


FIGURE  1 

Jfester  flow  chart  for  computer  simulation  in  which  system  is  characterized  bv 
what  happens  when  only  certain  important  events  occur. 
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If  the  next  earliest  occurring  event  in 
the  time  status  record  is  of  type  El 


1 

r 

No 

Is  the  operator  free? 

Yes 


♦ 


Add  1  to  the  queue 


i 


Note  operator  is 

not  free 

Determine  when  the  operator 
will  finish  working  on  this 
input:  Call  T(El)  the  time 
of  occurrence  of  the  event 
of  type  El.  Sample  from  the 
distribution  of  holding  times 
in  Table  II  and  obtain  a  time 
interval.  Obtain  the  time 
T(E2)  when  the  operator  will 
finish  by  adding  the  time 
interval  to  T(El).  Place  the 
T(E2)  thus  found  in  the  time 
status  record. 


— — - - * _ _ 

Determine  whether  the  input 
will  be  scrap.  Call  P  the 
probability  of  making  scrap. 
Since  there  was  no  queue  when 
the  operator  started  process¬ 
ing  this  input  P  is  found  to 
be  .10  from  Table  III  for  this 
demonstration.  Draw  a  two- 
digit  random  number.  If  the 
number  is  equal  to  or  less  than 
100P ,  then  the  operator  will 
produce  scrap,  and  this 
fact  should  be  noted. 


— —  i 

Remove  the  time  of  occurrence  of  this 
event  of  type  El  from  the  time  status 
record  and  find  the  next  occurring 
event  in  the  time  status  record. 


FIGURE  2 


If  the  next  occurring  event  in  the 
time  status  record  is  of  type  E2 
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5o  Draw  the  flow  charts,  showing  how  inputs  are  to  be  handled 
and  what  is  to  happen  when  particular  events  occur.  For 
those  systems  or  those  components  where  the  investigator 
cannot  write  the  rules  to  cover  all  interesting  possibil¬ 
ities,  then  a  computer  model  is  not  an  appropriate  model. 

6.  Program  a  suitable  computer  to  follow  the  instructions 
contained  in  the  flow  charts. 

7.  Obtain  the  component  characteristics  that  must  go  into  the 
model.  For  example,  distributions  of  arrival  or  holding 
times  must  be  obtained  either  by  measurement  or  by  theo¬ 
retical  studies.  Frequently  more  effort  goes  into  deter¬ 
mining  component  characteristics  than  in  any  other  part 
of  a  system  simulation  program. 

8.  Design  experiments  to  be  run  on  the  model  of  the  system. 

While  this  was  not  demonstrated,  many  costly  computer 
runs  may  be  avoided  by  proper  use  of  statistics  in  deter¬ 
mining  the  most  efficient  way  to  answer  the  questions 

to  be  answered, 

9-  Replicate  experiments  on  the  computer  a  number  of  times 
as  required  to  obtain  results  of  required  accuracy.  Some¬ 
times  as  few  as  ten  replications  will  peimit  an  estimate 
of  the  mean  that  meets  the  accuracy  requirements,  but  prob¬ 
ably  several  hundred  replications  will  be  necessary  to 
estimate  distributions, 

IX),  Devise  experiments  to  test  the  ability  of  the  model  to 
predict  system  performance. 

Not  all  of  the  above  steps  can  be  demonstrated  here',  but  all  are  impor¬ 
tant,  Space  does  not  permit  discussion  of  each  step  despite  the  importance 
of  each. 

The  purpose  of  the  rest  of  this  discussion  will  be  to  show  how  flow  charts 
can  be  drawn  for  more  complex  systems,  taking  into  account  more  factors  such 
as  equipment  reliability,  feedback,  and  series  and  parallel  operation.  While 
the  system  to  be  considered  next  is  only  slightly  more  complex  than  that  al¬ 
ready  shown,  no  attempt  will  be  made  to  show  how  instructions  are  followed 
and  how  inputs  are  traced  through  the  system  because  of  the  large  number  of 
pages  that  would  be  required.  Also,  no  system  of  really  interesting  complexi¬ 
ty  will  be  considered  because  merely  presenting  the  flow  charts  for  an  interest¬ 
ing  job  shop  scheduling  problem  or  for  an  interesting  air  defense  system  would 
also  require  too  many  pages.  However,  the  flow  charts  for  the  slightly  more 
complex  system  will  give  a  better  feeling  for  the  complexity  of  these  flow 
charts  and  the  extent  to  which  the  same  set  of  flow  charts  can  be  applied 
to  different  systems, .  It  will  be  clear,  for  example,  that  changing  compo¬ 
nent  characteristics  is  possible  in  going  from  one  set  of  runs  to  another 
and  that  the  flow  charts,  therefore,  have  considerable  generality.  That  is 
in  some  cases  the  flow  charts  for  an  air  defense  system  may  be  useful  with-* 
out  change  for  studying  missile  systems  and  manned  interceptor  systems. 
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It  will  also  be  e-vident,  however,  that  changing  rules  of  operation  require 
changing  the  flow  charts;  and  this  is  the  extent  to  which  the  flow  charts 
lack  generality.  If  the  alternative  rules  of  operation  of  interest  are 
known  beforehand,  it  is  possible  to  program  the  computer  so  rules  can  be 
readily  changed. 


THE  SIMPLE  SYSTEM  CHOSEN  FOR  ILLUSTRATION  OF  FLOW  CHARTING. 


The  figure  above  shows  a  simple  system  that  can  be  used  to  illustrate 
in  what  sense  a  system  can  be  characterized  by  what  happens  at  the  occurrence 
of  certain  important  events.  The  figure  is  also  useful  for  discussing  the 
component  characteristics  and  rules  of  operation  of  the  system  that  must  be 
known  if  the  system  is  to  be  studied  by  the  use  of  a  computer  simulation. 

The  figure  might  be  the  block  diagram  for  a  system  consisting  of  two 
typists  and  two  proofreaders.  The  typists  receive  rough  drafts  of  one  page 
reports  containing  considerable  tabular  data.  The  proofreaders  check  the 
numberical  data  in  the  reports  for  errors  in  typing. 

When  new  work  arrives  for  the  typists,  it  is  either  handed  to  a  free 
typist  or  is  placed  in  the  in-basket  if  both  typists  are  busy.  Both  typists 
take  work  from  the  same  in-basket. 
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When  a  typist  finishes  typing  a  report,  she  either  hands  the  report  to 
one  of  the  proofreaders  if  one  is  free  or  places  it  in  the  in-basket  shared 
by  the  proofreaders. 

There  are  two  types  of  reports.  Type  1  has  priority  over  type  2,  but 
occasionally  a  typist  or  a  proofreader  will  select  a  type  2  report  from  the 
in-basket  even  though  both  types  of  report  are  present  in  the  in-basket. 

Occasionally^  a  proofreader  finds  an  error  and  returns  a  report  to  the 
typists  for  retyping.  No  priority  is  given  to  reports  that  require  retyping. 

Occasionally  a  typewriter  fails  or  a  proofreaders  lamp  fails.  There  is 
one  maintenance  man.  If  equipment  fails  while  the  maintenance  man  is  busy, 
it  must  wait  for  service.  The  maintenance  man  selects  at  random  one  of  the 
requests  for  service  when  he  is  free. 

When  a  typewriter  fails,  the  typist  removes  the  report  from  her  typewriter 
and  places  the  rough  draft  in  the  in-basket.  That  is,  the  report  will  have 
to  be  completely  retyped  by  herself  or  the  other  typist.  When  a  proofreader's 
lamp  fails *  *bhe  proofreader  places  the  report  on  which  he  was  working  in  the 
in-basketo  That  is,  the  proofreading  will  have  to  be  done  over  by  either  one 
of  the  proofreaders. 


Work  placed  in  the  in-baskets  following  equipment  failure  is  not  given 
priority  over  other  work. 

The  paragraphs  just  above  show  the  amount  of  description  necessary  for 
even  a  simple  system.  Obviously  other  rules  of  operations  could  have  been 
chosen,  but  the  rules  described  above  permit  the  drawing  of  flow  charts  of 
only  modest  complexity. 

,,  Although  we  have  described  a  system  involving  typists  and  proofreaders, 
the  flow  charts  will  be  applicable  for  answering  some  questions  about  a  sys¬ 
tem  consisting  of  two  radar  operators  and  two  weapon  assigners.  Stations  1 
and  2  could  represent  two  operators  who  are  charged  with  identification  and 
tracking- of  aircraft.  If  new  blips  appear  on  the  scopes  while  both  operators 
are  busy,  they  must  wait  for  service.  Stations  3  and  1*  could  represent  two 
officers  charged  with  threat  evaluation  and  weapon  assignment.  When  both 
officers  are  busy,  newly  identified  targets  must  wait  for  service.  Occasion¬ 
ally  weapon  assigners  find  obvious  mistakes  in  identification  and  return  tar¬ 
gets  to  the  radar  detectors  for  re-entry  into  the  system.  If  equipment  fails 
while  any  one  of  the  operators  is  working  on  an  input,  the  input  must  return 
to  the  waiting  line  and  be  completely  reserviced  by  another  operator.  Prior¬ 
ity  of  targets  could  be  based  on  position.  Not  included  in  the  flow  charts 
to  follow,  however,  are  the  instructions  for  carrying  out  the  computation 
of  positions  of  aircraft  0 
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Typical  questions  that  might  lead  to  modeling  the  system  ares 

1.  How  will  daily  production  of  reports  be  affected  by  better 
training  of  typists?  Presumably  the  training  will  result 

in  a  different  distribution  of  service  times  for  the  typists 
and  a  lower  probability  of  making  an  error* 

2,  How  long  will  it  take  to  get  reports  out  if  we  double  the 
number  of  reports  to  be  typed  each  day?  Presumably,  in¬ 
creasing  the  rate  of  arrival  or  otherwise  changing  the  dis¬ 
tribution  of  arrivals  will  affect  the  time  spent  by  reports 
in  waiting  for  service* 

3*  If  we  buy  electric  typewriters,  what" will  be  the  cost  of 
producing  a  finished  report?  Presumably  new  typewriters 
are  being  considered  because  they  will  shorten  the  typist 
service  times  and  lower  the  probability  of  error.  The  cost 
of  a  finished  report  can  be  determined  from  the  usual  costs 
plus  the  cost  of  new  equipment  suitably  amortized. 

These  and  other  questions  that  might  be  asked  require  a  means  for  find¬ 
ing  the  relationship  between  the  followings 

1.  Distributions  of  arrival. 

2.  Percentage  of  priority  reports. 

3.  Probability  of  operator  making  a  selection  error. 

lu  Probability  of  an  operator  spoiling  an  input* 

3>.  Distributions  of  operator  holding  times.  Note  that  3*  U* 
and  5>  will  be  affected  by  training  and  equipment  provided 
the  operator. 

6*  Rules  for  selecting  inputs  from  among  those  in  waiting  lines. 

7*  Distributions  of  breakdowns  of  equipment. 

8*  Distributions  of  repair  times* 

9*  Distributions  of  waiting  times  and  times  required  to  get 
inputs  through  the  system. 

10,  Costs  of  wages,  floor  space,  material,  power,  and  equipment. 

A  digital  computer  simulation  will  permit  finding  the  relationships  be¬ 
tween  the  parameters  listed  above. 

We  will  now  proceed  to  the  flow  charts  for  the  system.  They  will  not 
contain  instructions  for  printout  but  will  emphasize  the  tracing  of  inputs. 
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NOTATION  AND  ABBREVIATIONS  IN  THE  FLOW  CHARTS. 

To  assist  in  the  understanding  of  the  flow  charts  that  follow,  the  abbre¬ 
viations  and  notation  used  in  the  flow  charts  will  be  explained. 

jEL  Is  the  event  of  the  arrival  of  an  input  of  type  j. 

If  j  equals  1,  the  input  is  of  higher  priority  than 
if  j  equals  2. 

ijE2  Is  the  event  that  either  station  1  or  station  2  has 

just  completed  processing  an  input  of  type  j.  Thus, 
i  reifers  to  either  station  1  or  station  2. 

kjE3  Is  the  event  that  either  station  3  or  station  1;  has 

just  completed  processing  an  input  of  type  j.  Thus, 
k  refers  to  either  station  3  or  to  station  lu 

sEl;  Is  the  event  of  a  breakdown  of  station  s  where  s  can 

refer  to  any  one  of  the  four  stations. 

sE5  Is  the  event  that  station  s  has  just  been  repaired 

by  the  maintenance  man. 

T(ijE2)  Is  the  time  of  occurrence  of  the  event  ijE2,  and  the 

time  of  occurrence  of  one  of  the  other  events  is 
designated  in  a  similar  way. 

WLI  Is  the  waiting  line  for  inputs  to  stations  1  and  2. 

WL2  Is  the  waiting  line  for  inputs  to  stations  3  and  1*. 

V1LM  Is  the  waiting  line  for  repair  jobs  waiting  for  the 

maintenance  man. 

While  most  of  the  instructions  in  the  flow  charts  are  self  explanatory, 
some  may  require  explanation.  For  exaxrple. 

Generate  a  T(ijE2) 

stands  for  the  following  set  of  instructions  t  Call  the  time  of  occurrence 
of  the  event  of  type  jEl  present  time.  Sample  from  the  distribution  of 
service  times  for  this  station  and  obtain  a  time  interval.  Add  this  time 
interval  to  present  time  to  find  the  time  when  this  station  will  finish  pro¬ 
cessing  the  input. 

The  question 

Error? 

stands  for  the  following  instruction:  Let  p  be  the  probability  the  station 
will  make  an  error  in  selection  of  an  input.  Draw  a  random  number  from  a 
population  rectangularly  distributed  between  zero  and  one.  If  the  random 
number  is  less  than  p,  say  the  station  made  an  error.  If  the  random  num¬ 
ber  is  larger  than  p,  say  the  station  did  not  make  an  error  in  selection. 
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The  instruction 

TSR 

stands  for  the  following  set  of  instructions:  Delete  from  the  time  status 
record  the  time  of  occurrence  of  the  event  for  which  the  subroutine  was  just 
executed.  Find  the  next  occurring  event  in  the  time  status  record  and  exe¬ 
cute  the  subroutine  associated  with  this  event  type. 

Figure  I|.  contains  instructions  for  handling  inputs  to  the  typists.  In¬ 
puts  are  immediately  serviced  if  one  of  the  typists  is  free  and  go  into  the 
waiting  line  if  neither  of  the  typists  is  free. 

Figure  5  shows  the  instruction  for  disposing  of  an  input  processed  by 
the  typists  and  also  contains  instructions  for  simulating  reaching  for  new 
work  if  jobs  are  waiting. 

Figure  6  shows  the  instructions  for  simulating  the  handling  of  outputs 
from  the  proofreaders  and  also  contains  instructions  for  simulating  the  reach¬ 
ing  for  new  work  on  the  part  of  a  proofreader. 

Figure  7  shows  the  instructions  for  handling  inputs  when  one  of  the 
stations  breaks  down  because  of  equipment  failure. 

Figure  8  gives  instructions  for  simulating  repair  and  shows  how  to  handle 
inputs  when  equipment  has  been  repaired. 

Note  in  Figure  7  that  a  breakdown  leads  to  removing  an  event  of  type 
ijE2  or  kjE3  from  the  time  status  record.  Thus,  some  of  the  events  in  the 
time  status  record  arepirely  tentative. 


Figure  U 
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FIGURE  5 
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As  the  flow  charts  indicate,  five  event  types  will  be  stored  from  time 
to  time  in  the  time  status  record*  The  times  of  occurrence  of  events  of  type 
EL  can  be  generated  before  the  simulation  begins.  Also,  whether  a  particular 
input  is  a  priority  input  can  be  determined  in  advance  from  the  known  proba¬ 
bility  of  occurrence  of  a  priority  input.  If  a  large  number  of  inputs  are 
to  be  studied  and  there  is  insufficient  room  in  storage  to  accommodate  an 
the  times  of  arrival,  then  it  is  possible  to  generate  only  some  of  the  arrival 
times  before  the  simulation  and  generate  the  rest  one  by  one  as  times  of  occur¬ 
ence  of  these  events  are  deleted  from  the  event  list  or  time  status  record* 

Repaired  equipment  can  be  expected  to  operate  for  some  length  of  time 
before  breakdown  occurs  again.  The  time  to  break  down  is  a  distributed 
quantity.  The  first  time  of  occurrence  of  the  event  type  sEl*  for  each  station 
can  be  obtained  by  sampling  from  the  distribution  of  time  to  failure  for 
each  station.  Thus,  before  the  simulation  begins  there  will  be  four  events 
of  type  sEU  in  the  time  status*  record.  Times  of  occurrence  of  succeeding 
breakdowns  cannot  be  determined  until  equipment  has  been  repaired* 

Before  the  simulation  begins  it  is  necessary  to  know  and  read  into 
the  computer  the  distributions  and  probabilities  that  describe  component 
characteristics. 

As  stated  before,  no  attempt  was  made  to  include  in  the  flow  charts  instruc¬ 
tions  for  printing  out  information.  .  Potentially,  any  of  the  blocks  in  the 
flow  charts  could  call  for  print  outs.  In  Figure  U,  for  example,  instructions 
could  call  for  the  computer  to  print  out  each  time  an  operator  becomes  free 
or  becomes  busy.  Such  information  may  be  desired  in  attempting  to  determine 
what  fraction  of  time  operators  are  busy.  Or  the  computer  might  be  required 
to  print  out  the  size  of  the  waiting  line  each  time  an  input  is  added  or  re¬ 
moved  from  the  waiting  line  if  information  is  desired  concerning  the  average 
length  of  the  waiting  line  or  the  distribution  of  the  lengths  of  the  waiting 
line.  So  much  information  can  be  printed  out  that  the  investigator  must  be 
careful  to  resist  the  temptation  to  "print  out  everything."  Information 
should  be  printed  out  only  when  there  is  a  definite  use  for  it.  Much  com¬ 
putation  time  can  be  wasted  in  printing  out  needless  quantities,  and  the  in¬ 
vestigator  can  soon  be  engulfed  with  masses  of  data  that  nobody  wants  to 
interpret. 

In  looking  back  over  the  examples,  I  am  struck  by  the  fact  that  I  in¬ 
cluded  no  examples  of  the  types  of  technical  computations  usually  required 
to  represent  some  of  the  components.  These  computations  are  frequently 
associated  with  up-dating  the  positions  of  inputs  or  components  when  position 
is  important.  Also,  in  simulating  military  systems,  such  computations  are 
required  to  determine  trajectories  of  missiles,  for  example,  and  iiqpact  points 
oi"  interception  points 0  Thus,  the  time  of  occurrence  of  an  event  may  depend 
on  a  technical  computation  as  well  as  the  result  of  sampling  from  a  distri¬ 
bution. 

No  special  section  will  be  devoted  to  utilization  of  models.  The  remarks 
leading  to  the  development  of  flow  charts  necessarily  contained  considerable 
discussion  of  the  uses  to  which  mathematical  models  are  put* 
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The  examples  have  shown  that  a  wide  variety  of  system  components  can  be 
represented  by  numerical  quantities  and  rules  for  computation  to  solve  many 
system  problems.  There  still  remains  many  system  problems  that  cannot  be 
solved  using  the  techniques  we  have  discussed  mostly  because  we  do  not  know 
how  to  represent  components  or  do  not  know  the  rules  of  operation. 


COMPUTER  PLAYED  WAR  GAMES 

Jo  Jo  Dunne  and  R0  A.  Sebastian* 

Ballistic  Research  Laboratories 
Aberdeen  Proving  Ground,  Maryland 

The  Weapon  Systems  Laboratory  of  the  BRL  uses  war  games  as  one  of 
several  means  of  evaluating  army  weapons  and  weapon  families.  The  games 
are  designed  to  be  played  automatically  on  the  ORDVAC**. 

,  The  ORDVAC  is  a  high-speed  electronic  digital  computer.  It  was  built 
for  the  BRL  several  years  ago  by  the  University  of  Illinois.  It  is  a  gen¬ 
eral  purpose  computer  which  may  be  used  for  solving  many  varieties  of 
mathematical  problems. 

In  addition  to  the  four  elementary  operations  of  addition,  subtraction, 
multiplication,  and  division,  the  computer  performs  other  basic  operations. 
The  most  important  of  these  are  (i)  duplicating  numbers,  (ii)  moving  num¬ 
bers  from  one  part  of  the  machine  to  another,  and  (iii)  after  testing  two 
quantities  for  "equality"  or  for  a  "greater  than"  condition,  choosing  one 
of  two  paths  to  follow  depending  on  the  results  of  the  comparison.  Every 
problem  to  be  computed  on  the  machine  must  in  the  final  analysis  be  ex¬ 
pressed  in  terms  of  the  basic  operations. 

The  ORDVAC  memory  is  composed  of  two  devices,  the  magnetic  core  with 
a  capacity  of  1*096  words  and  the  magnetic  drum  with  a  capacity  of  10,032 
words.  In  the  memory  we  can  store  data  and  information  which,  in  case  of 
hand  computations,  are  kept  in  the  mind,  or  recorded  in  notes,  tables,  etc. 
We  can  store  not  merely  the  numerical  results  of  all  operations,  but  also 
a  sequence  of  orders.  Thus,  the  computer  can  be  automatically  sequenced 
so  that  it  can  perform  a  whole  program  of  computations  without  any  human 
intervention.  The  sequence  in  turn  needs  a  detailed  set  of  instructions— 
the  code— taking  into  account  all  the  acts  of  judgment  and  memory  that 
a  person  would  perform  in  carrying  out  the  computations  by  hand. 

When  needed,  the  computer  can  easily  generate  random  numbers  distri¬ 
buted  uniformly,  normally,  or  according  to  any  simple  distributions. 

The  operating  speed  of  the  ORDVAC  is  quite  high.  Add  time  is  f?0 
microseconds  (i.e.  about  20,000  additions  per  second)  and  multiplication 
and  divide  times  are  about  3 00  microseconds. 

Shift  time  and  storage  access  time  of  the  core  memory  are  about  1 $ 
microseconds.  The  magnetic  drum  is  much  slower  and  is  used  when  there  is 
not  enough  storage  space  in  the  core  memory. 


Well,  so  much  for  the  computer  used  in  our  war  games.  Next  I  would 
like  to  take  up  the  procedure  used  in  designing  games  to  be  played  on  the 
ORDVAC. 

Several  steps  are  involved  here  and  the  overall  time  required  depends 
on  the  complexity  of  the  game.  For  games  designed  to  date  this  procedure 
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has  required  about  one  man  year,  but  for  more  complex  games  it  might  take 
several  man  years  to  complete. 

The  first  step  is  to  define  the  problem  to  be  investigated  and  to 
determine  what  information-output  data— is  desired  from  the  game.  The 
Weapon  Systems  Laboratory,  of  course,  is  interested  primarily  in  questions 
concerning  weapons  and  families  of  weapons,  so  that  our  games  are  designed 
with  emphasis  on  weapons’  effectiveness. 

The  second  step  in  designing  a  game  is  to  prepare  a  battlefield  scenario 
describing  the  two  opposing  forces— their  weapons,  targets,  logistics,  and  in¬ 
telligence— and  the  tactical  situation— the  time  frame,  outstanding  terrain 
features,  lines  of  departure  and  contact  and  military  policy  and  objectives. 

Based  on  the  battlefield  scenario,  a  detailed  set  of  rules  is  then 
provided.  These  rules  must  be  in  sufficient  detail  that  play  can  be 
carried  out  for  all  conceivable  situations  that  might  arise  during  a  play 
of  the  game.  These  rules  would  include  all  the  input  data  on  weapons, 
targets,  logistics,  intelligence,  movement,  terrain,  and  how  these  data 
vary  with  time  and  circumstances. 

At  various  points  in  the  play  of  the  game  the  commanders  would  be 
presented  with  choices  from  several  alternatives  and  would  make  their 
choices  taking  into  account  all  the  information  available  at  that  moment. 

In  computer  played  war  games,  all  the  choices  that  could  arise  must  be 
provided  for  before  play  of  the  game  commences,  since  it  would  be  imprac¬ 
tical  to  continually  start  and  stop  the  computer  during  the  play  of  the 
game  in  order  to  make  the  decisions  as  they  arise. 

Finally,  the  game  must  be  coded  so  that  it  can  be  played  automati¬ 
cally  on  the  OEDVAC.  This  step  takes  several  months  to  complete.  Because 
of  the  complexity  of  war  games  most  of  the  core  memory’s  4096  words  are 
used  as  storage  space.  It  takes  from  3  to  10  minutes  to  play  a  game  once 
on  the  ORDVAC  and  it  is  usually  necessary  to  play  the  game  several  hundred 
times  to  obtain  all  the  desired  information.  Here,  again,  the  times  in¬ 
volved  depend  on  the  complexity  of  the  game. 


Next,  I  would  like  to  describe  the  war  games  that  we  have  designed  to 
date .  So  far  the  War  Games  Group  has  devoted  its  effort  to  games  played 
at  two  different  levels— the  company  level  and  the  army  level. 

The  game  at  the  company  level  was  designed  for  use  in  comparisons  of 
various  infantry  rifles.  Play  begins  with  the  Red  assault  force  moving 
out  from  the  line  of  departure.  Physically,  the  line  of  departure  might 
be  a  road  or  stream.  The  objective  of  the  Red  force  is  a  terrain  feature, 
such  as  a  hill  or  a  wooded  area.  The  objective  is  assumed  to  be  located 
about  1500  feet  from  the  line  of  departure. 

Red  moves  forward  under  the  cover  of  artillery  in  several  waves,  one 
wave  moving  for  a  short  period  of  time  while  the  others  take  cover  and 
give  it  protective  fire.  The  effect  of  Red’s  fire  is  to  inflict  casualties 
on  the  Blue  defender  and  to  suppress  Blue’s  small  arms  fire.  Blue  exchanges 
artillery  and  small  aims  fire  with  Red.  The  effect  of  Blue’s  fire  is  to 
inflict  casualties  on  Red  and  to  force  Red  to  advance  at  a  faster  rate. 
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When  Red  has  advanced  to  about  1*00  feet  from  the  Blue  force.  Red's 
artillery  fire  is  lifted  because  of  friendly  safety  requirements#  When 
they  have  advanced  to  about  120  feet  from  the  Blue  force  the  several  Red 
waves  combine  into  one  wave  of  men  and  advance  at  a  constant  speed  of  7-1/2 
feet  per  second#  Red  now  fires  from  the  hip,  while  Blue  continues  to  em¬ 
ploy  aimed  rifle  fire«  During  this  final  stage  of  the  assault,  hand  gre¬ 
nades  are  also  exchanged# 

At  the  start  of  each  game  there  are  128  Blue  riflemen#  The  size  of  the 
Red  force  is  varied  from  game  to  game  and  may  be  as  high  as  6I4.O  men#  In 
order  to  facilitate  the  play  of  the  game  on  the  ORDVAC,  the  play  is  carried 
out  in  128  independent  engagements,  with  one  Blue  soldier  opposed  by  the 
appropriate  number  of  Red  soldiers# 

For  each  engagement,  a  record  is  kept  of  the  ranges  at  which  Blue 
and  Red  exchange  rifle  fire  and  hand  grenades,  and  the  ranges  at  which 
Red  and  Blue  are  killed  by  rifle  fire,  artillery  fire,  and  hand  grenades. 

The  records  of  128  engagements  are  combined  to  give  the  cumulative  per¬ 
centages  of  Blue  and  Red  casualties  against  range,  and  the  cumulative 
number  of  rifle  shots  and  hand  grenades  exchanged  by  Blue  and  Red  against 
range# 


The  game  ends  when  one  side  or  the  other  suffers  a  sufficiently  high 
percentage  of  casualties#  From  cumulative  graphs  of  the  Blue  and  Red 
casualties  the  ORDVAC  can  determine  the  range  at  which  the  game  ends, 
which  side  wins  the  game,  the  number  of  casualties  suffered  by  each  side, 
and  the  number  of  shots  and  grenades  exchanged  by  Blue  and  Red. 

Several  factors  are  taken  into  account  in  the  play  of  the  game: 

(1)  Terrain 

(2)  Red's  speed  and  maneuver 

(3)  Size  of  the  targets 

(4)  Rifle  characteristics  -  lethality,  accuracy  and  rate  of  fire 

(5)  Effects  of  artillery  fire 

(6 )  Effects  of  hand  grenades 

(7)  Number  of  waves  of  Red  soldiers,  and 

(8)  The  ratio  of  the  Red  force  to  the  Blue  force# 

All  together  14  groups  of  25  games  each  are  played  for  the  same  rifle. 

The  ratio  of  the  Red  force  to  the  Blue  force  is  the  quantity  that  is  varied 
from  one  group  of  games  to  the  next#  Of  particular  interest  is  the  frac¬ 
tion  of  games  won  by  Blue  as  a  function  of  this  ratio.  Curves  showing  the 
probability  that  Blue  will  win  vs#  the  ratio  of  Red  to  Blue  can  be  constructed 
for  each  rifle  type  and  used  in  comparisons  of  the  different  rifles. 
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When  this  game  was  being  designed  it  was  feared  that  differences 
between  rifles  would  be  masked  by  other  effects,  if  the  game  was  made 
too  complex  by  including  many  infantry  weapons,  such  as  machine  guns, 
automatic  rifles,  tanks,  and  so  on.  For  this  reason  we  included  only 
rifles,  artillery  mortars,  and  hand  grenades,  and  even  then  we  expected 
moderate  differences  to  be  lost.  To  date,  about  2000  games  have  been 
played  employing  six  different  rifles  having  various  accuracies,  leth¬ 
alities,  and  rates  of  fire,  and  actually,  moderate  variations  in  rifle 
characteristics  have  resulted  in  appreciable  differences  in  the  outputs. 

This  is  illustrated  in  the  figure.  The  center  curve  was  obtained 
for  the  Ml  cal.. 30  rifle.  The  right  hand  curve  was  obtained  for  a  rifle 
10#  more  accurate  and  lethal  than  the  Ml,  and  the  left  hand  curve  was 
obtained  for  a  rifle  10#  less  accurate  and  lethal  than  the  Ml. 

You  notice  that  in  the  area  of  interest  (in  which  each  side  has  some 
chance  of  winning)  the  curves  for  the  different  rifles  are  parallel.  For 
this  reason  the  ratio  at  which  each  side  has  an  equal  chance  of  winning 
is  a  good  measure  for  comparing  the  rifles*  effectiveness.  It  is  seen 
from  the  figure  that  a  10#  change  in  rifle  characteristics  results  in 
about  a  10#  change  in  the  rifle's  effectiveness. 

In  games  played  at  the  army  level,  one  Blue  army  opposed  a  Red  force 
of  equal  or  greater  strength.  Studies  at  this  level  are  conducted  by  the 
War  Games  Group  for  one  of  three  reasons? 

(1)  To  compare  two  families  of  weapon  systems 

(2)  To  compare  two  weapon  systems 

(3)  To  show  a  need  or  lack  of  need  for  a  certain  type  of 
weapon  system. 

Two  basic  models  are  used  in  these  studies.  The  first — The  Ground 
War  Game— was  patterned  after  a  model  devised  by  Northrop  Aircraft,  Inc. 
under  contract  to  BRL;  the  second  — ANSWEG#  was  recently  completed  by  the 
War  Games  Group.  One  of  the  major  differences  in  the  two  models  is  that 
ANSWEG  uses  Monte  Carlo  methods  while  the  Ground  War  Game  uses  average  or 
expected  values  for  all  numbers. 

In  these  models,  a  time  period  such  as  one  hour  or  one  day  is  estab¬ 
lished  and  the  computer  executes  the  various  steps  in  this  time  period 
over  and  over.  (In  ANSWEG  the  main  steps  are:  Intelligence,  Non-atomic 
Battle,  Atomic  Battle,  and  Movement.)  One  of  the  difficult  problems  is  to 
decide  how  much  detail  to  put  into  each  step.  More  detail  perhaps  makes 
for  a  better  model,  but  it  also  can  begin  to  tax  the  capacity  of  the  ma¬ 
chine  as  well  as  the  human  operator  who  is  burdened  with  the  task  of 
gathering  a  large  amount  of  input  data.  Moreover  more  detail  in  one  step 
can  demand  more  time  and  more  input  data  in  another  step.  A  good  example 
of  this  can  be  found  in  the  Atomic  Battle  step  of  ANSWEG.  Both  the  Red 
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and  Blue  forces  possess  a  number  of  targets  that  are  attacked  by  the  enemy 
with  atomic  weapons.  Similar  targets  are  grouped  together  into  a  single 
target  type.  Now  it  seems  logical  to  assume  that,  up  to  a  point,  more 
target  types  result  In  a  better  model.  However,  more  target  types  also 
increase  the  machine  time  of  the  game  and  the  amount  of  input  data  since, 
for  every  target  type,  the  following  questions  must  be  answered* 

(1)  How  easy  is  it  for  the  enemy  to  find  one  of  these  targets? 

(2)  What  enemy  weapon  systems  will  fire  at  these  targets? 

(3)  How  do  these  targets  move  on  a  battlefield? 

(4)  When  one  of  these  targets  is  killed,  how  is  the  performance  of 
that  side  affected? 

The  last  question  has  been  the  most  difficult  to  answer  even  for  the 
most  obvious  atomic  targets.  It  is,  of  course,  an  easy  matter  to  state 
the  actual  function  of,  say,  a  communications  center.  But  when  a  center 
for  one  side  is  "knocked  out"  there  must  be  a  provision  in  the  model  to 
somehow  penalize  that  side.  If  there  is  no  such  provision,  the  enemy  will 
be  better  off  in  the  game  not  to  fire  at  these  targets. 

As  was  mentioned  before,  one  of  the  purposes  of  these  war  games  is 
to  compare  two  families  of  weapon  systems.  This  is  done  by  running  a 
game  (or  series  of  games  with  identical  inputs)  with  the  characteristics 
of  one  family  in  the  model*  and  then  running  another  game  with  the  char- 
acteristics  of  the  other  family.  Nothing  else  but  these  characteristics 
is  changed  from  the  first  game  to  the  second.  However,  for  a  given  study 
more  than  one  pair  of  games  is  usually  compared.  Factors  such  as  level 
of  intelligence  and  available  supply  of  atomic  warheads  are  varied  from 
pair  to  pair. 


Unfortunately,  it  is  not  usually  possible  to  run  a  set  of  games  and 
obtain  as  a  result  two  numbers  which,  when  compared,  will  reveal  the  better 
family.  One  family  will  undoubtedly  show  up  better  by  one  criterion  and 
the  other  family  will  be  better  by  another  criterion.  Some  of  the  criteria 
that  have  been  used  ares 


(1)  Cost  of  launchers,  missiles,  and  warheads  expended 

(2)  Amount  of  fissionable  material  expended 

(3)  Weight  of  missiles  fired 

(4)  Numbers  of  casualties 

(5)  Number  of  miles  advanced 

(6)  Number  of  days  the  war  lasts. 
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If  war  games  were  the  only  method  used,  it  would  indeed  be  difficult 
to  make  the  final  decision  as  to  which  family  is  more  desirable.  However, 
war  gaming  is  only  one  of  several  methods  used  by  the  BRL  to  arrive  at  a 
final  decision. 

In  planning  for  new  generations  of  war  games,  a  number  of  problems 
still  remain  unsolved  by  the  War  Games  Group,  The  basic  problem  is  this: 
How  can  war  games  be  made  both  simple  and  realistic?  Specific  problems 
are: 


(1)  How  can  logistics,  intelligence,  mobility,  and  terrain  be 
introduced  into  computer  played  war  games  in  a  realistic  way?  and 

(2)  How  can  strategy  and  tactics  be  introduced  realistically  into 
the  games  and  still  keep  the  number  of  battlefield  decisions  at  a  minimum? 
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ABSTRACT  ♦  The  required  pace  of  Weapon  System  Development,  the  unit  cost 
of  flight  tests,  and  the  engineering  complexity  of  future  missile  systems,  ' 
present  a  problem  in  maintaining  a  sufficiently  complete  and  current  compre¬ 
hension  of  the  technical  nature  of  a  system  among  those  who  require  such  loiow— 
ledge.  Testing  is  viewed  as  the  primary  source  of  information  about  a  develop¬ 
ing  system.  Mathematical  models  are  proposed  as  the  routine  vehicle  for  the 
current  understanding  of  the  system,  and  are  represented  as  the  heart  of  the 
testing  operation.  Present-day  "dete  minis  tic'"  flight  simulation  models  are 
briefly  described.  Future  probablistic  models  are  discussed.  The  over-all 
problem  of  advancing  test  capability  is  suggested  for  more  thorough  OR  treat¬ 
ment. 


lo  INTRODUCTION.  In  recent  years  we  have  witnessed  a  rapid  growth  of 
the  so-called  "system  concept”  or  "system  viewpoint"  in  various  fields  of 
engineering.  The  national  challenge  for  efficient  development  of  effective 
weapons  has  lent  special  impetus  to  the  evolution  of  this  concept.  Of  course, 
the  nation's  weapon  and  space  vehicle'  pro jects  have  been  most  notably  in-  * 
fluenced  by  its  application, 

Basic  to  the  system  concept  is  the  emphasis  on  interactions  and  inter¬ 
dependencies  of  the  component  subsystems.  The  engineering  objective  in 
general  is  to  create  or  adapt  components  which,  when  made  to  perform  -together 
produce  a  timely,  economical,  and  reliable  fulfillment  of  the  intended  system 
purpose. 


Maintaining  a  working  comprehension  of  this  purpose  and  of  the  component 
interactions,  during  the  development  of  a  large,  complex  system,  presents 
a  formidable  problem  to  the  developing  agency.  Component  design  activities 
require  a  comprehensive  knowledge  about  the  system  as  Well  as  detailed 
knowledge  about  adjacent  components .  Program  managers' and  decision  makers 
require  concurrent  information  concerning  system  capability,  cost,  and 
availability.  There  are  many  activities  which  require  current  information 
about  the  growing  system  in  terms  of  logistics,  safety,  training,  compati¬ 
bility  with  other  systems,  etc.  This  general  problem  of  maintaining  a 
working  comprehension  of  the  system  and  its  purpose  among  participating 
agencies  according  to  need,  invites  extensive  attention  and  deliberate 
application  of  tenets  not  unlike  those  associated  with  the  field  of  opera¬ 
tions  research. 

While  Operations  Research  has  done  much  for  missile  system  planning, 
the  author  believes  that  it  can  also  speed  up  the  missile  system  develop¬ 
ment  process  by  contributing  to  improved  test  operations.  Extensive  use 
of  mathematical  models  as  "vehicles"  for  the  current  and  working  under¬ 
standing  of  each  large  system,  and  incorporation  of  advance  in-flight 
measurement  techniques  constitute  the  major  part  of  the  needed  improvement. 
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The  evaluation  of  test  operation,  of  course,  provides  the  quantitative 
data  for  each  system.  Admittedly,  much  of  the  missile  development  process 
is  what  has  been  called  in  OR  literature  "ill-structured,"  i.e.,  not 
amenable  to  known  OR  computational  techniques.  However,  good  OR  work  is 
characterized  by  consideration  first  of  the  problem  rather  than  the 
technique.  A  potential  OR  field  is  introduced  here  together  with  some 
proposed  concepts  and  a  brief  discussion  of  some  of  the  associated  technical 
research  currently  underway. 

The  present  work  has  arisen  incidental  to  the  Ordnance  Mission  of 
Missile  Systems  Testing  at  the  White  Sands  Missile  Range,  New  Mexico. 

2.  TESTING  PROVIDES  THE  FEEDBACK.  Intelligent  system  management  and 
program  decisions  require  measurements  of  the  developing  system  that  continu¬ 
ously  estimate  the  capability,  suitability,  and  reliability  of  the  system. 
Test  results  of  the  designed  hardware  serve  as  sound  bases  for  the  needed  es¬ 
timates  for  both  subsystem  and  overall  system  tests. 

In  general,  testing  involves  evaluation  and  system  analysis.  Evalua¬ 
tion  determines  the  quantitative  information  characteristic  of  the  realized 
system  and  its  parts.  Systan  analysis  determines  the  functions  and  inter¬ 
actions  of  the  system  parts.  Improved  methods  of  expressing  and  maintain¬ 
ing  the  current  understanding  of  complex  systems  is  therefore  an  essential 
part  and  product  of  an  advanced  test  operations. 

The  missile  system  development  process  can  be  briefly  outlined  as 
follows:  (1)  an  agency  initiates  a  problem  statement  based  on  a  need  and 
support  of  an  authority,  (2)  research  discovers  and  creates  methods  and 
devices  to  realize  the  desired  system  and  subsystem  capabilities,  (3)  testing 
reveals  existing  inadequacies  within  subsystems  and  between  subsystems  and 
further,  suggests  solutions  for  removing  the  inadequacies,  and  (h)  after 
all  modifications  are  completed  the  system  is  operable  thus  providing  the 
solution  to  the  problem  initiated  by  the  agency. 

The  problem  statement  mentioned  above  usually  consists  of  a  compilation 
of  the  desired  capabilities  and  characteristics,  together  with  a  functional 
block  diagram  illustrating  what  the  system  is  to  do,  and  the  basic  inter¬ 
connections  between  the  subsystems.  Progress  in  subsystem  design  has  con¬ 
siderable  influence  on  the  problem  statement. 

Perhaps  the  most  vital  output  of  the  test  program  is  the  feedback 
information  essential  to  the  efficient  progress  of  the  design  and  develop¬ 
ment  stage.  The  feedback  information  must  lend  a  stability  and  direction 
to  the  process.  Thus,  in  the  system  development  process,  testing  is  the 
feedback. 

To  aid  visualization.  Figure  1  breaks  down  the  system  into  blocks  with 
division  into  system  tests  and  subsystem  tests.  Component  and  laboratory 
tests  will  be  considered  as  subsystem  tests.  Figure  1  also  shows  closed 
loop  connections  of  these  tests  with  the  mathematical  model  and  its  read¬ 
out  device,  the  simulation  computer. 
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FIGURE  1 
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3«  MISSILE  SYSTEM  MATHEMATICAL  MODELS.  To  conduct  the  missile  system 
test  operation  an  "understanding''  and  definition  of  the  tested  system  is 
necessary.  Generally,  in  the  past  this  understanding  has  been  largely  in 
the  mind  of  the  test  engineer.  It  may  be  expressed  in  different  forms  in¬ 
cluding  equations,  diagrams,  graphs,  etc.,  but  it  must  be  more  and  more  speci¬ 
fied  and  definitive  in  order  to  accommodate  and  promote  advanced  and  accelerat- 
ing  methods  of  missile  and  space  vehicle  development  0  It  must  be  expressed 
as  much  as  possible  in  quantitative  or  mathematical  terms.  The  model  of  the 
system  provides  a  productive  and  useful  end  for  the  numbers  which  are  evaluated 
from  the  test  measurements.  It  also  integrates  the  otherwise  piece-meal  in¬ 
formation  about  the  system. 

The  model  is  the  heart  of  the  test  operation.  It  begins  existence  as 
the  functional  block  diagram  which  is  constructed  in  the  problem  statement 
stage  of  the  development  process.  It  can  be  used  in  the  subsystem  design 
stage  for  obtaining  optimal  design  parameters  and  for  determining  allowable 
component  tolerances.  Fig.  2  indicates  the  inputs  to,  and  uses  of  the  model. 
Every  test  experience  with  the  developing  system  should  be  directed  to  up- 
dating  and  improving  the  missile  system  mathematical  model.  The  current  model  f 


Figure  2 


A  GROWING  EXPLICIT  UNDERSTANDING  OF  THE  MISSILE  SYSTEM  IS  CENTRAL 

TO  THE  SYSTEM  TEST  OPERATION 
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in  turn,  is  instrumental  in  designing  the  tests  and  interpreting  their  results. 
Based  on  the  understanding  of  the  system,  studies  may  be  completed  which  set 
forth  the  needed  tests,  the  data  required,  and  the  accuracies  required. 

Although  this  sounds  like  a  bootstrap  procedure,  it  is  actually  a  process  of 
learning,  in  which  new  facts  are  based  on  the  old.  In  this  case,  the  memory 
which  is  kept  up-to-date  according  to  hardware  test  experience  is  comprehensive 
and  less  volatile.  It  is  defined  with  respect  to  assumptions,  tentative 
information,  and  approximations.  It  is  accesible  in  detail  for  study,  and 
also  is  readily  adaptable  to  computer  mechanization  for  solution  of  system 
performance  and  of  over-all  capabilities  and  limitations. 

Improvements  in  the  format  for  the  mathematical  model  and  in  the  pro¬ 
cedures  for  its  use,  of  course,  are  the  subject  for  continual  research. 
Experience  at  the  White  Sands  Missile  Range  has  involved  several  types  of 
diagrams  including  verbal  block  diagrams  in  which  descriptive  words  are  used 
to  label  data  flow  and  subsystem  function,  and  mathematical  block  diagrams 
in  which  transfer  functions  are  displayed.  A  possible  development  will  be 
the  '‘correspondence  table"  in  which  each  system  variable  and  parameter  is 
listed  together  with  complete,  appropriate  information  concerning  its  loca¬ 
tion  in  the  actual  field  hardware,  its  mode  (Electrical  modulation,  voltage 
range,  etc.),  its  noise  characteristics,  etc.  Opposite  this  information  in 
the  table  would  be  corresponding  information  with  respect  to  the  mathematical 
simulation  of  the  variable  (including  precision,  approximations, etc.)  A  third 
column  of  information  may  be  included  in  the  table  concerning  the  transducer 
and  measurement  characteristics  of  the  variable. 

It  is  emphasized  that  the  purpose  of  the  model  is  served  primarily  dur¬ 
ing  the  development,  i.e.,  while  the  system  is  growing  and  changing.  A 
final  complete  mathematical  model  is  no  longer  useful  for  its  primary  intended 
purpose. 

Associated  with  needed  research  toward  improved  formats  and  procedures 
in  connection  with  the  mathematical  model,is  that  for  models  which  efficiently 
convey  the  test  output  information  to  the  various  recipients.  Considerable 
attention  must  be  given  to  defining  measures,  for  example,  of  the  system 
merit  which  are  appropriate  to  higher  command,  measures  of  component  suffi¬ 
ciency  appropriate  to  designers,  and  measures  of  maintainability,  operability, 
and  safety,  for  the  user. 

As  procedures  become  more  clearly  defined,  perhaps  along  the  lines 
suggested  by  Figure  2,  the  entire  operation  submits  itself  more  and  more 
to  automation.  It  is  tempting  to  extrapolate  to  the  future  situation  wherein 
immediately  at  the  conpletion  of  each  flight  test,  all,  the  information  in¬ 
herent  in  the  completed  test  is  contained  in  the  up-dated  mathematical  model. 

lj..  MTSSILE  SYSTEM  MATHEMATI  CAL  M3  DELS ,  Mathematical  models  of  missile 
systems,  as  applied  in  engineering  testing,  have  been  somewhat  different 
from  the  models  common  in  operations  research.  The  purpose  of  simulation, 
i.e.,  of  exercising  models,  is  the  same,  however.  The  purpose  is  to  generate 
"runs,"  or  simultaneous  histories  of  the  variables,  which  are  consistent  with 
known  or  assumed  inputs  and  consistent  with  characteristics  and  interactions 
of  the  component  subsystems.  The  "runs"  may  be  intended  to  show  over-all 
system  characteristics  as  they  result  from  the  input  and/or  component  characte¬ 
ristics,  or  they  may  be  intended  to  reveal  the  effects  upon  the  component 
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subsystem  as  they  result  from  the  desired  over-all  system  performance.  Com¬ 
puter  programming  of  the  model  enables  convenient  experimentation  and  study 
of  the  system. 

Mathematical  flight  simulation  has  grown  primarily  from  the  mechaniza¬ 
tion  of  integro-differential  equations  on  analog  computers.  The  design  of 
servo-mechanisms  and  automatic  control  systems  has  for  many  years  involved 
simulation  techniques,  both  for  analysis  of  observed  performance  and  for 
synthesis  of  proposed  characteristics.  Dynamic  analogies  have  been  employed 
wherein  electrical,  or  other  convenient  physical  quantities,  are  made  to 
respond  according  to  the  same  mathematical  equations  as  those  which  govern 
the  response  of  the  simulated  system.  Thus,  in  missile  flight  simulation, 
the  dynamics,  the  motions  consistent  with  the  physical  forces  and  reactions, 
are  generated  through  mathematical  analogies.  The  components  of  the  analog 
computer  can  be  considered  as  mathematical  operators  (integrators,  multi¬ 
pliers,  etc.).  Similarly,  a  given  component  of  a  missile  system  may  be  con¬ 
sidered  to  operate  upon  its  inputs  to  provide  its  respective  outputs. 

The  response  of  a  fin  servo,  an  autopilot,  a  tracking  radar,  or  an 
accelerometer,  for  example,  are  simulated  through  operational  analogies  de¬ 
scribed  by  transfer  functions.  In  addition,  extensive  missile  kinematics 
are  included  in  the  present-day. flight  simulations.  That  is,  the  continuous 
computation  of  quantities  such  as  position,  angles,  velocities,  accelera¬ 
tions,  etc.,  as  they  appear  in  different,  relatively  moving  reference  frames, 
are  included  to  enhance  the  utility  of  the  simulation  model.  The  type  of 
mathematical  model  being  described,  and  presently  in  wide  use,  is  called 
" deterministic"  or  '‘rigid."  Its  solutions  are  repeatable  and  it  does  not, 
in  general,  contain  stochastic  variables.  jMbre  realistic  and  more  useful 
models  are  being,  developed  in  which  the  stochastic  element  is  inserted.  The 
ultimate  model  will  provide  statistical  system  experience,  through  simulated 
flights,  which  will  closely  predict  actual  missile  system  in-flight  relia¬ 
bility  and  intercept  dispersions. 

Steps  from  the  purely  deterministic  model  toward  the  so-called  "proba- 
balistic"  model  are  envisioned  and  some  are  already  achieved.  The  determi¬ 
nistic  model  is  used  to  find  the  sensitivity  to  parameter  deviations  in 
what  are  called  perturbation  studies.  A  more  advanced  achievement  is  repre¬ 
sented  by  the  successful  insertion  of  characteristic  noise  in  the  determi¬ 
nistic  model  at  one  point  to  find  out  how  it  affects  other  points  in  the 
system  or  to  study  the  manner  in  which  the  noise  propagates  throughout  the 
system.  Such  noise,  for  example  radar  tracking  noise,  "unclean"  booster  se¬ 
paration  or  thrust  termination,  airframe  flexure,  etc.,  has  been  reproduced 
or  approximated  from  component  or  system  test  data.  Figure  3  illustrates 
how  closely  this  type  of  simulation  resembles  the  observed  results  obtained 
from  telemetered  records  of  field  flights.  Real  flight  f  in  motions  in  this 
case  closely  resemble  those  simulated  when  typical  radar  noise  is  inserted, 
open-loop,  into  the  siirmlated  system0  The  next  steps  to  be  taken  would  $ 
appear  to  include  closed-loop  generation  of  the  environmental  variables, 
ttien  to  include  simulated  susceptibilities  of  components  to  the  environment. 
Figure  h  displays  the  probabalistic  model  which  generates  missile  perfor¬ 
mance,  generates  coherent  in-flight  environment,  and  which  feeds  back  sta¬ 
tistical  component  susceptibility  in  terms  of  failure  or  of  degraded  per¬ 
formance.  A  final  addition  might  be  an  "initial  condition"  block  in  which 


FIGURE  4. 

AM  ILLUSTRATION  OF  A  SYSTEM  RELIABILITY  MODEL 
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calibrations  done  by  humans  before  the  flight,  component  susceptibilities  as 
they  depend  on  preflight  handling,  storage,  etc.,  are  selected  from  appro¬ 
priate  probability  distributions. 


Since  simulation  serves  as  a  powerful  tool  in  synthesizing  or  integrat¬ 
ing  knowledge  of  component  subsystem  characteristics  toward  knowledge  of 
over-all  system  characteristics,  it  is  felt  that  research  in  quest  of  the 
probabalis tic  model  will  contribute  significantly  to  the  existing  problem 
of  improving  in-flight  reliability  of  missiles  and  space  craft.  Simulation 
developments  should  complement  concurrent  developments  in  environmental 
testing  equipments  and  in  advanced  flight  test  techniques.  Again,  the  model 
can  be  the  basic  tool  with  respect  to  designing  and  interpreting  flight  and 
laboratory  tests. 

>  REAL-TIME  FLIGHT  ANALYSIS  AND  TEST  CONTROL.  The  modeling  approach 
to  missile  system  testing  provides  for  a  positive  communication  and  identifi¬ 
cation  of  system  characteristics.  It  contributes  to  efficient  direction  of 
complex  system  developments.  And,  as  has  been  mentioned,  it  prepares  the 
way  for  increased  automation.  Each  test  flight  of  a  large  missile  system 
inherently  contains  a  considerable  amount  of  information  about  the  system. 

Time  and  cost  considerations  force  us  to  glean  as  much  of  the  inherent  in¬ 
formation  as  possible  from  each  flight  test  experiment.  The  time  involved 
gleaning  the  information  obtainable  from  a  missile  flight  (or  hardware 
test  of  any  kind)  and  determining  its  meaning  with  respect  to  system  ob¬ 
jectives  is  an  influential  part  of  tie  technical  lead-time  associated  with 
the  weapon  development.  Automation  quite  certainly  will  reduce  this  time 
more  and  more0  In  addition  to  this,  however,  there  is  a  Mstep  advantage  *' 
which  will  accrue  when  automation  arrives  at  the  point  allowing  flight  analy¬ 
sis  and  test  control  to  be  conducted  in  real-time,  namely  during  the  flight. 
It  is  the  advantage  of  influencing  the  course  of  the  experiment  according 
to  the  results  of  the  analysis  and  evaluation,  and  according  to  the  immediate 
in-flight  circumstances. 

Reference  is  made  to  Figure  5.  The  privilege  of  selecting  monitored 
missile  variables  to  be  telemetered  according  to  need  would  increase  the 
efficient  utilization  of  flight  test  data  links.  Also,  real-time  data  re¬ 
duction  such  as  missile-borne  frequency  analysis  would  increase  the  amount 
of  data  obtainable  per  unit  of  data  link  capacity. 

The  automatic  improvement  of  the  mathematical  model  through  real-time 
closed-loop  adjustment  of  the  simulated  parameters  is  visualized  with  the 
aid  of  Figure  6.  Consider  a  flight  test  in  which  the  primary  objective  is 
to  automatically  adjust  certain  specified  coefficients  in  the  mathematical 
model  according  to  the  performance  being  manifest  in  the  actual  flight.  The 
idea  is  basically  simple  in  that  the  flight  analysis  and  test  control  facility 
compares  measured  quantities  in  the  flying  system  with  corresponding  quanti¬ 
ties  generated  by  the  missile  system  simulator.  -Detected  differences  are 
employed  as  error  signals  in  adjusting  servo  mechanisms  which  cause  the  simu- 
lated  coefficients  to  assume  the  values  corresponding  to  their  effective 
counterparts  in  the  flying  system.  The  analysis  and  control  facility  employs 
control  on  the  flying  system  to  acheive  appropriate  test  conditions.  For 
example,  an  experiment  planned  at  WSMR,  in  which  given  aerodynamic  coeffi¬ 
cients  are  to  be  evaluated,  will  employ  commands  to  the  missile  which  serve 
to  increase  the  resolution  of  the  computations.  It  may  be  desirable  to  force 
conditions  such  that  selected  mathematical  terms  are  made  equal  to  zero  while 
a  given  measurement  is  being  made. 
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Fig.  5 

REAL-TIME  FLIGHT  DATA  PROCESSING 


•  EARLIER  TEST  RESULTS 

•  INCREASE  INFORMATION  CAPACITY  OF  DATA  LINKS 
•  CLOSED-LOOP  MODEL  IMPROVEMENT 


MISSILE  SYSTEM 

Dynamic  Test  Sequencing 
Malfunction  Diagnosis 
In-flight  Design  Adjustments 
Alternate  Usage  of  "Failed"  Flights 


TEST  SYSTEM 

In-flight  Instrumentation  Control 
Drone  Vectoring 

Instrumentation  for  Acquisition 
and  Tracking 
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The  automatic  improvement  of  the  simulated  system  is  a  great  step  nearer 
an  ultimate  capability  wherein  every  experience  witii  the  'system  hardware  clear¬ 
ly  extends  the  understanding  of  the  system  and  results  directly  in  conclusions 
concerning  the  next  step  in  the  weapon  development  cycle* 

The  last  item  listed  in  Figure  5  refers  to  the  application  of  real-time 
flight  analysis  for  the  purpose  of  test  control.  Note  that  the  control  of 
the  test  may  be  directed  at  the  missile  system  which  is  under  test  or  it  may 
be  directed  at  the  range  system  which  is  performing  the  test  and  measurement. 
Illustrative  of  the  latter  case  are  the  assigning  and  assisting  of  data  acqui¬ 
sition  instruments  and  the  vectoring  of  drone  craft  as  targets.  The  items 
associated  with  controlling  the  tested  system  are  of  primary  interest  and  are 
those  which  excite  the  imagination  most  strongly  from  the  point  of  view  of 
gaining  more  system  data  from  the  test  flight  through  advanced  system  test 
techniques.  The  opportunity  to  influence  the  sequence  of  test  phases  as  the 
flight  proceeds,  increases  the  potential  output  of  normal  flights.  The  privi¬ 
lege  of  initiating  a  s.eries  of  diagnostic  commands  which  immediately  localize 
a  malfunction  and  furnish  additional  data  before  the  missile  is  lost  is  also 
important.  Future  advances  may  allow  certain  remedial  measures  and/or  redi¬ 
rection  of  the  flight  test  to  achieve  alternative  objectives.  Many  of  these 
items  take  on  a  greater  significance  when  considered  in  connection  with  very- 
expensive  future  testing  of  space  craft. 

6*  CONCLUDING  REMARKS.  The  discussion  has  briefly  described  certain  re¬ 
searches  and  how  they  fit  together  in  a  possible  forward-looking  prospective 
of  advanced,  missile  system  testing.  The  field  requires  much  further  study 
and  action  in  order  to  maintain  a  system  testing  capacity  commensurate  with 
the  radically  new.  dimensions  of  future  weapon  systems.  It  is  felt  that  the 
over-all  problem  is  appropriate  for  treatment  by  operations  research.  Certain 
resulting  conclusion  may  have  a  beneficial  inpact  on  existing  R&D  policies, 
such  as  perhaps  far-reaching  measures  toward  insuring  that  a  proposed  weapon 
system  be  "testable."  However,  it  is  anticipated  that  the  greater  inpact 
acceleration  of  meaningful  research  and  implementation  of  ad¬ 
vanced  test  techniques. 


DISCOUNTED  LEAST  SQUARES 


R. 


J.  Duff  in1  and  Th.  W.  Schmidt2 


This  note  describes  a  method  of  extrapolation  of  data  by  fitting  with 
polynomial  functions.  The  method  permits  use  of  polynomials  of  arbitrary 
degree.  However,  for  the  sake  of  being  definite,  let  us  suppose  that  the 
fitting  is  done  with  second  degree  polynomials  of  the  form 

2 

(l)  p(x)  =  aQ  +  axx  +  a2x  . 


It  is  supposed  that  the  data  consists  of  observations  given  at  a  regular 
sequence  of  values  of  x,  say  x  =  1,  2,  3,  The  observations  for  these 
values  of  x  are  denoted  by  y-^,  y^,  jy  ...  Then  the  central  problem  is  to 

extrapolate  this  sequence  of  observations  to  obtain  a  predicted  value  at 
x  =  0.  This  extrapolated  or  predicted  value  will  be  denoted  by  y^.  Thus 
if  the  polynomial  p(x)  is  suitably  fitted  to  the  data,  the  extrapolated 
value  is  given  by  y*  =  aQ. 


A  common  method  of  fitting  is  to  select  an  integer  r, 
and  to  employ  a  least  squares  fit  of  p(x)  by  minimizing 

r 

(2) 


termed  the  range. 


E  &  -  p(nD 


Instead  of  this  procedure,  we  minimize  the  expression 

(3)  E  =  \  ^n  "  p(n^j  2‘ 

We  term  this  method  discounted  least  squares .  The  constant  0  is  termed  the 
discount  factor.  In  a  qualitative  way  the  discount  factor  in  (3)  corresponds 
to  the  range  in  (2).  Like  the  range,  the  discount  factor  must  be  selected 
to  suit  best  the  character  of  the  data.  This  selection  requires  judgment. 
However,  typical  values  would  be  r  *  20  and  0  =  0.8. 

The  notion  of  discounted  least  squares  was  suggested  by  certain  problems 
in  the  analysis  of  missile  trajectory  data. 3  For  example,  from  a  sequence  of 
observations  relating  to  the  position  of  a  missile  it  might  be  desired  to 
predict  the  point  of  impact.  In  such  applications,  x  corresponds  to  negative 
time,  and  the  sequence  y  is  usually  termed  a  time  series. 


^Director  of  Special  Research  in  Applied  Mathematics,  Duke  Unviersity 
and  Office  Of  Ordnance  Research,  Durham,  North  Carolina 
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Chief,  Systems  Research  Office,  Office  of  Ordnance  Research,  United 
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”A  simple  formula  for  prediciton  and  automatic  s cnltation , ”  by 
R.  J.  Duffin  and  Th.  W.  Schmidt,  presented  to  the  American  Rocket 
Society,  Flight  Testing  Conference,  Daytona  Beach,  Florida,  March  24, 
1959. 
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The  method  of  discounted  least  squares  is  a  general  method,  and  we 
believe  it  should  have  various  applications  besides  that  of  rocket  trajec¬ 
tory  analysis.  In  particular  it  should  suit  certain  problems  of  economics 
and  operations  research  for  which  the  progressive  discount  of  the  past  seems 
natural. 

When  the  minimization  of  (3)  is  carried  out,  a  formula  of  the  following 
type  results: 


(4) 


7° 


Vn 


Here  denotes  coefficients  which  depend  on  6  but  not  on  the  sequence  yn< 

We  term  (3)  the  long  formula.  Actually  the  long  formula  is  infinitely  long, 
but  the  coefficients  Q  decay  exponentially  and  ar$  sensibly  zero  for  large 
n.  This  also  indicatei  that  practically  the  sequence  y  does  not  have  to 
be  infinite.  Roughly  speaking,  the  practical  length  o^the  long  formula  is 
a  small  multiple  of  the  range  when  using  method  (2). 

The  coefficients  Q  have  an  explicit  algebraic  expression.  This 
expression  will  not  be  given,  because  in  most  cases  there  is  an  alternative 
prediction  formula  which  has  many  advantages  over  (4).  Thus  the  short 
formula  is 


(5) 


yo =  ^yi +  “  3^y2 +  ®2y +  (y3 +  * 


Here  *  y^  -  yk  is  termed  a  discrepancy.  By  y^  we  mean  the  predicted  value 
of  yk  based  on  the  previous  values  yk+2»  •••  The  numerical  coeffi¬ 

cients  in  (5)  are  simply  the  binomial  coefficients  of  (a  -  b)3.  More  gen¬ 
erally,  if  the  prediction  had  been  based  on ''cubic  polynomials,  then  the 
coefficients  in  (5)  would  be  obtained  from  (a  -  b)4. 


Formula  (5)  is  called  the  "short  formula"  because  the  predicted  value 
is  given  as  a  function  of  the  last  three  observed  values  and  the  last  three 
discrepancies.  The  "shortness"  is  a  consequence  of  using  memory.  The  mem¬ 
ory  is  selective  because  only  the  pertinent  six  numbers  are  used  rather 
than  the  whole  infinite  time  series. 


An  important  feature  of  the  short  formula  not  manifestly  evident  is 
that  prediction  can  now  be  carried  as  far  into  the  future  as  desired.  For 
example,  suppose  that  y,  is  actually  not  known.  Then  in  the  formula  y,  is 

x  *  1 
simply  replaced  by  the  predicted  value  y^.  The  general  rule,  then,  is  that 

if  the  observed  values  are  not  known,  they  are  to  be  replaced  by  predicted 
values . 
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To  employ  the  short  formula  it  is  first  necessary  to  select  a  starting 
index,  say  k0  Thus  y^  is  to  be  the  first  predicted  value.  The  values  of 

the  discrepancies  &k+^,  S^,  311(1  appearing  in  the  short  formula  are 

not  known.  To  get  started  let  us  assume  them  to  be  equal  to  zero.  This  is 
equivalent  to  assuming  that  the  values  of  y  for  n>k  are  given  by  a  certain 
quadratic  polynomial.  This  method  of  starting  has  the  desirable  property 
of  giving  perfect  prediction  if  the  observations  y  are  actually  defined  by 
a  quadratic  polynomial.  n 

There  is  no  difficulty  in  coding  the  short  formula  for  a  digital  com¬ 
puter.  Moreover,  provisions  can  be  made  in  this  coding  for  automatic 
scrutation.  By  scrutation  we  mean  inspection  of  data  for  the  purpose  of 
rejection  of  obvious  errors  or  blunders.  In  hand  calculation,  scrutation 
is  often  accomplished  by  looking  over  a  graph  of  the  data.  The  trend  in 
data  processing  is  toward  elimination  of  human  judgment.  To  this  end  a 
method  of  employing  the  short  formula  as  an  automatic  scrutator  is  as 
follows.  Suppose  that  by  theoretical  or  empirical  means  an  estimate  can 
be  made  of  a  standard  deviation  of  y^,  say  o(y^) .  Then  if 

(6)  |Sn|>3°n 

we  consider  y  to  be  a  blunder,  and  y  is  replaced  by  y  .  The  factor  3  is 
n  n  n 

merely  a  conventional  value. 

A  common  phenomenon  in  data  processing  is  a  missing  or  lost  observation. 
In  automatic  scrutation  a  lost  observation  is  treated  as  an  "excess  error." 
Thus  if  yfi  is  lost  it  is  replaced  by  y*,  and  the  program  proceeds  without 

interruption. 

In  the  coding,  provision  must  be  made  for  the  contingency  that  the 
discrepancy  repeatedly  exceeds  the  prescribed  tolerance.  To  do  this  an 
integer  w  is  selected.  If  at  any  time  relation  (6)  is  satisfied  for  w 
consecutive  values  of  n,  then  all  past  memory  is  wiped  out  and  a  fresh 
start  is  made.  This  process,  coded  into  the  computer,  would  permit  ex¬ 
trapolation  of  curves  made  up  of  parabolic  segments.  An  application  would 
be  the  tracking  of  evasive  action. 

Besides  gross  errors  it  is  necessary  to  take  into  account  the  effect 
of  small  errors.  To  understand  this,  consider  a  sequence  y^  such  that 

7n  =  0  for  all  n  except  that  yk  =  1.  We  may  regard  y^  as  a  unit  error 

deviating  from  the  true  values  zero.  Substituting  this  sequence  in  the 
long  formula  gives  y^  =  Q^.  The  same  relation  would  result,  of  course, 

if  we  used  the  short  formula,  because  the  short  formula  is  merely  an 
identity  which  the  long  formula  satisfies.  Thus  it  is  seen  that  the 
coefficients  determine  how  a  single  error  is  propagated  into  the 

future.  Moreover,  the  coding  of  the  short  formula  gives  an  automatic 
method  of  numerically  evaluating  the  coefficients  Q  . 
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A  knowledge  of  the  sequence  0  is  necessary  for  the  analysis  of  random 
errors  in  all  the  observations  as  well  as  the  single  error  discussed  above. 
Thus  suppose  that  all  the  observations  have  independent  random  errors  with 
the  same  variance.  Then  the  variance  in  the  prediction  is  reduced  by  a 
factor  £  Q~  .  This  sum  approaches  zero  as  0  approaches  one.  (Increasing 
the  degreeof  the  polynomial  tends  to  increase  the  sum.)  In  favorable 
cases  one  obtains  appreciable  smoothing  as  well  as  extrapolation. 


A  PRACTICAL  APPLICATION  OF  THE  WAITING  ILNE 
THEORY  FOR  THE  OPTIMUM  ALLOCATION  OF  MACHINE 
REPAIR  OPERATORS 

J.  Moundalexis 
Industrial  Engineer 
Arsenal  Operations  Division 
Picatinny  Arsenal 

INTRODUCTION.  This  paper  will  deal  with  the  practical  aspects  of 
applying  the  Waiting  Line  Theory  to  a  problem  of  determining  the  optimum 
number  of  machine  repairmen  required  to  repair  machines  which  breakdown 
in  a  random  manner. 

Random  demand  activities  arise  in  such  areas  like  a)  gasoline  stations 
where  customers  arrive  in  a  random  manner,  b)  toll  booth  operations  or  c) 
people  arriving  at  a  restaurant. 

The  object  in  allocating  people  to  such  type  activities  becomes  one 
of  determining  the  optimum  number  of  people  required  to  handle  a  given 
workload.  The  aim  may  be  a  minimum  cost  program  also  taking  into  account 
certain  elements  which  are  not  readily  definable  quantitatively. 

The  usual  method  of  establishing  reliable  labor  performance  standards 
on  routine  repetitive  activities  has  been  solved  by  using  direct  stop 
watch  study  and  summarizing  the  times  for  each  work  element.  However  in 
the  area  of  random  demand  activities  the  method  for  establishing  personnel 
manning  requirements  is  usually  based  upon  arbitrary  judgment.  The  progress 
in  the  practical  application  of  the  Waiting  Line  Theory  to  random  .demand 
activities  has  been  unfortunately  limited,  probably  due  to  the  reluctance 
of  practitioners  to  get  involved  in  the  preparation  of  mathematical  models 
for  their  own  specific  situations.  It  is  important  that  work  in  the  util¬ 
ization  area  be  continued  and  common  reason  be  applied  to  analyze  the  net 
worth  of  a  given  situation  by  developing  and/or  using  waiting  line  models 
as  is  practical  and  logical. 

GENERAL  DESCRIPTION  OF  PROBLEM.  The  problem  under  discussion  is  to 
determine  the  optimum  number  of  repairmen  required  to  keep  machines  in 
operation  to  assure  that  production  line  downtime  cost  losses  as  a  result 
of  machine  breakdown  are  a  minimum.  The  area  under  discussion  is  the 
Ammunition  Loading  area  where  approximately  40  subassembly  lines  are  con¬ 
tinually  in  operation  making  complex  and  difficult  manufactured  ammunition 
ranging  from  illuminating  powder  packed  in  a  container  to  the  explosive 
loading  of  missile  warheads.  This  complex  involves  at  any  one  time  approx¬ 
imately  200  people  and  65  machines  and  is  located  within  an  approximate  one 
mile  square  area.  For  practical  reasons,  in  view  of  the  nature  of  the  work¬ 
load  and  security  restrictions,  the  total  area  is  sub-divided  into  several 
homogeneous  sub-areas.  A  sample  of  two  sub-areas  is  shown  in  EXHIBIT  1„* 


*  Exhibits  can  be  found  at  the  end  of  this  article 
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SCHEMATIC  ILXUSTRATIOK  OF  THE  PROBLEM.  The  problem  deals  with  a 
multiple  channel -infinite  universe.  The  schematic  representation  of  this 
model  is  shown  in  EXHIBIT  2.  The  schematic  shows  the  arrivals,  machines 
waiting  for  service  with  a  mean  arrival  rate  a  (Exponential  Distribution) . 

The  mean  arrival  rate  is  equal  to  one  over  the  mean  time  between  machine 
breakdown.  These  arrivals  (Mq)  are  being  placed  in  the  service  channels 
on  first  come  first  served  basis.  The  service  channels  denoted  by  (Rq) 
represent  repairmen  available  to  make  the  repair.  The  mean  service  rate 
is  denoted  by  (y)  which  is  equal  to  the  reciprocal  of  the  mean  service 
time  (o).  Thus  7T  m  l/o. 

MATHEMATICAL  FORMULATION.  Several  expressions  of  the  multiple  Channel  - 
Infinite  Universe  model  used  to  solve  this  problem  are  as  follows  i 

Where  M  *  Number  of  machines 

R  =  Number  of  repairmen 

o  ■  Average  machine  service  time 

it  ■  Average  time  between  each  machine  breakdown 
o 

it  =  S  =  Machine  repair  &  performance  index 

Pn(t)  =  Probability  of  n  machines  requiring 
service  at  a  time  (t) 

Pn('k)  “  ^o^n!  * 

-p0(t)  Jl _ 

R!  R(n-R) 

Since  <*=> 

E  p  (t)  »  1 
m»o  n 

E(wait)  =  —  E  (n-R)  p  (t) 
n  n=R+l  n 

ILLUSTRATED  COMPUTATION  PROCEDURE.  A  sample  illustrated  computation 
procedure,  average  machine  and  repairman  wait  times,  for  a  problem  having 
$  machines  with  one  repairman  is  shown  in  EXHIBIT  3.  It  is  evident  that 
hand  computation  is  entirely  too  cumbersome,  for  higher  value  problems  and 
therefore,  for  those  who  have  need  for  such  tables,  reference  is  made  to 
the  Machine  -  Serviceman  Table  prepared  by  the  Scientific  and  Industrial 
Computing  Laboratory,  Kodak  Park  Works,  Kodak  Eastman  Company  or  other 
available  sources. 

DEFINITION  OF  PROBLEM.  The  problem  data  of  sub-area  A  includes  thirty 
machines  with  a  ratio  of  three  operators  per  machine.  The  mean  service  time 
is  1  hour  while  the  mean  time  between  machine  breakdown  is  10  hours.  The 
machine  and  repair  index  is  0.1.  The  wait  time  values  for  machines  and 
repairmen  using  different  numbers  of  repairmen  is  shown  below: 


n  <  R, 
n  >  R. 
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WAIT  TIME  SCHEDULE 
AMMUNITION  LOADING  PRODUCTION  LINE 
_ _ SUB-AREA  A 


Nr.  of 

Repairmen 

Utilized 

Repairmen 

Wait  Time 

Machine 
Wait  Time 

1 

.0000 

.6333  or 
63.33  % 

2 

.0106 

.271*5 

3 

.1536 

.0689 

1* 

.3299 

.0172 

5 

.1*230 

.0106 

6 

.51*60 

.0011 

8 

.6591 

.0000 

The  values  in  the  table  indicate  that  if  two  repairmen  are  utilized 
to  service  the  30  machines  then  the  average  repairman  wait  time  is  1.06% 
while  the  machine  wait  time  awaiting  repair  is  27*15%.  Obviously  +.hjg 
result  should  be  translated  into  something  tangible. .  In  this  instance 
it  is  desired  to  express  the  percentage  figure  of  each  category  in  a 
dollar  equivalent  to  determine  the  minimum  cost  program.  As  was  stated 
a  machine  down  for  repairs  idles  an  average  of  three  operators.  This 
expressed  in  dollars  means  that  a  1%  average  machine  wait  (based  on  an 
eight  hour  shift)  is  equivalent  to  a  total  loss  of  $27  per  shift.  By 
referring  to  the  table  below*  the  cost  program  for.  the  each  repairmen 
crew  size  can  be  seen. 


Nr.  of  Repairmen 

COST  PROGRAM 

30  machines 

Cost  of  Repair 
(operator  $25/day) 

Down  Time 

Loss 

Total 

1 

$25 

$1*710 

$1,735 

2 

5o 

71*1 

791 

3 

75 

186 

261 

1* 

100 

1*6 

11*6 

5 

125 

29 

151* 

(cont.) 
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COST  PROGRAM  (cont.) 


30  machines 

Cost  of  Repair  Down  Time 

Nr.  of  Repairmen  (operator  $25/day)  Loss  Total 

6  $150  $3  $153 

7  175  0  175 

8  200  0  200 

DISCUSSION.  The  ideal  minimum  cost  solution  to  this  problem  is  to 
utilize  h  repairmen  resulting  in  a  cost  program  of  $lU6/ day. 

However  if  the  repairmen  wait  time  can  be  utilized  for  a  preventive 
maintenance  program  perhaps  it  would  be  advantageous  assuming  the  avail¬ 
ability  of  2  extra  repairmen,  to  use  6  repairmen.  This  would  add  $7  to 
the  cost  program  but  would  yield  an  extra  equivalent  of  8  hrs.  for  pre¬ 
ventive  maintenance  (based  opon  60$  utilization  factor  of  wait  time). 

And  of  course  there  would  be  the  extra  1.61$  production  count. 

CONCLUSION.  There  are  many  applications  of  the  Waiting  Line  Theory 
models  to  random  demand  activities  within  the  Ordnance  Corps.  The  results 
will  be  fruitful  and  will  provide  the  basis  for  continual  improvement. 
Further  the  coupling  of  scientific  techniques  and  common  reason  will  yield 
practical  and  logical  solutions  to  difficult  problems. 
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SCHEMATIC  REPRESENTATION 
OF  A 

MULTIPLE  CHANNEL  -  INFINITE  UNIVERSE 


Machines 


ft  *  Mean  Service 
Rate 


CM  =  Mean  Arrival  Rate 


Service 

Channels 


where  C*rr)  is  the 
mean  time  between 
machine  breakdown 


16=0'  where  <&)  is  the 
mean  service  time 


Exhibit  2 


SAMPLE  COMPUTATION  PROCEDURE 


M  =  5  =  Nr  of  Machines 
R  =  1  =  Repairman 
S  =  0.1 


PM!  sn  . 

n  5  where 

Pn  *  **'  Sn  where 

P  "  n!  (M-n) !  R  >  n  >  0 

O  *" 

Po  R!  Rn-r(M-n) !  R<n 

p.  -  p  pi  -  (.565X.5) 

1  °r 

o 

p?  -  -  (.565)(.2) 

o— 

o 

P,  ■=  P ‘  fl  ■  (.565) (.06) 

^  °  p* 

O 

p4  -  P  -  (.565) (.012) 

Po 

pc  -  Pn^i  -  (.565) (.001) 

>  0<p 


.2825 

.1130 

.034 

.0067 

.0006 


Exhibit  3 
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Exhibit  3«  (cont.) 
Probabilities  for  M  =  5  8  =  0.1 
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n  n-R 


R-n 


0 

0 

1 

2 

3 

4 


1 

0 

0 

0 

0 

0 


.5650 

.2825 

.1130 

.0340 

.0067 

.0006 


Machine  Wait 
Time 


M 

■  E  (n-R)  P 
n=R+l 


n 


R  <  n 


£  (1)(.  1130)+(2) (. 0340)+(3) (.0067)  +  (4)(. 0006) 

5 

.04  or  4  7„ 


Repairman 
Wait  Time 


E  (R-n)  P 


n=o 


n 


n  <  R 


R 

(DC.565) 

1 


565  or  56.5% 
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LINEAR  PROGRAMME  M}  APPLIED  TO  ECONOMIC 
SELECTION  OF  MATERIALS  FOR  FURNACE  CHARGES 

Gideon  I,  Gardner 

General  Thomas  J,  Rodman  Laboratory 
Watertown  Arsenal 

ABSTRACT,  Many  combinations  of  furnace  input  materials  exist  which  will 
meet  a  set  of  given  furnace  output  requirements .  The  Linear  Programming  tech¬ 
nique  can  be  used  to  choose  that  feasible  material  combination  whose  material 
cost  is  a  minimum.  The  technique  can  be  used  to  calculate  optimum  prescrip¬ 
tions  for  individual  heats,  or  to  determine  which  materials  to  buy  to  stock 
in  inventory.  This  paper  is  concerned  with  the  "what  to  buy1'  decision;  a 

»  ,  *  .  a  _  ,  ,  on  a  high  speed  digital  computer,  and  several 

Duplications  relating  to  the  problem  are  discussed, 

INTRODUCTI ON 0  At  some  point  in  the  operation  of  a  foundry,  decisions 
must  be  reached  which  specify  the  particular  furnace  input  materials  (ferro¬ 
alloys,  bar  stock,  pig,  etc,)  to  be  purchased.  The  materials  which  are 
bought  are  eventually  drawn  from  inventory,  and  melted  together  to  form  a 
product  of  some  specified  composition  and  gross  weight.  It  is  unfortunately 
true  that  the  decision  concerning  "what  to  buy*'  is  often  made  without  consider¬ 
ation  as  to  the  infinite  number  of  possible  combinations  of  different  input 
types  (drawn  from  the  catalogues  of  the  many  furnace  input  producers)  which 
will .meet  the.  specified  requirements.  How  is  one  to  choose  among  these  many 
feasible  combinations?  Does  a  criterion  in  fact  exist  which  allows  these 
different  combinations  to  be  compared?  Obviously  yes,  since  each  feasible 
combination  will  generally  involve  a  different  overall  material  cost,  and 
this  cost  should  be  minimized.  It  will  be  shown  that  the  technique  called 
Linear  Programming  can  be  used  to  systematically  evaluate  the  many  possible 
combinations  of  inputs  according  to  the  least-cost  criterion,  thereby  allow¬ 
ing  the  solution  to  the  best-input-mix  problem  to  be  readily  obtained, 

LINEAR  PROGRAMMING,  Linear  Programming  has  previously  been  successfully 
applied  to  "best  mix  problems  as  described  above.  For  example,  a  Columbia 
economist  has  computed  the  least-cost  mix  of  foods  that  a  person  could  con¬ 
sume  and  yet  meet  all  the  accepted  nutritional  requirements!  farms  have  applied 
the  method  practically  in  the  determination  of  least-cost  cattle-feeds;  and 
the  petroleum  industry  in  particular  has  realized  substantial  cost  savings 
m  applying  L,  P,  (Linear  Programming)  to  gasoline  blending  problems,  3 

In  general  mathematical  terms,  the  technique  applies  to  those  processes 
or  activities  which  are  subject  to  a  set  of  linear  restrictions  in  the  form 
of  equalities  or  inequalities ,  If  the  number  of  feasible  solutions  within 
the  constraints  is  large,  L,  P,  can  choose  that  particular  solution  among 
all  feasible  ones  which  optimizes  an  objective  linear  function. 

The.  charging  of  metals  in  a  furnace  is  an  activity  which  fits  the  above 
description;  one  could  wish  to  choose  the  least-cost  mix  of  raw  materials 
which,  when  melted,  will  give  a  product  of  known  total  weight  and  chemical 
composition.  Since  each  raw  material  is  of  different  composition,  the  linear 
restrictions  are  those  which  relate  the  contents  of  each  chemical  element 
in  each  raw  material  to  the  required  content  of  that  element  in  the  end  pro¬ 
duct,  If  is  the  unknown  weight  to  be  charged  of  each  raw  material  i,  and 
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is  the  percent  by  weight  of  each  element  j  in  each  raw  material  i,  then 


where  R  is  the  total  weight  of  the  heat  and  Pj  is  the  required  percentage 

of  element  j.  The  equations  represented  by  the  above  symbology  are  in  fact 
element  balances ;  some  appear  as  equalities,  for  example  the  restriction 
that  manganese  content  equal  some  specified  amount;  others  appear  as  inequali¬ 
ties,  e.g.,  the  requirement  that  sulphur  (an  inpurity)  content  shall  be  less 
than  or  equal  to  some  specific  limit;  others  are  also  in  inequality  form,  but 
appear  as  part  of  a  two  equation  set  stating  that  the  final  content 
lie  within  a  specified  range. 


There  is  another  linear  restriction  which  relates  the  weight  of  each 
raw  material  charged  to  the  total  weight  of  the  heat. 


El.  «  H 
i  1 


Still  other  restrictions  appear  in  the  metal-charging  problem,  according  to 
the  particular  problem  being  solved.  For  example,  one  might  limit  the  utili¬ 
zation  of  specific  raw  materials  according  to  their  availability.  E.g.,  if 
home-scrap  is  used,  then  the  amount  generated  must  limit  the  amount  utilized. 

The  linear  restrictions  mentioned  above  make  up  a  set  of  simultaneous 
linear  equations  and/or  inequations*  with  the  I^'s  being  the  unknown  variables. 

Linear  Programming  extracts  an  optimum  solution  to  the  above  set  of  equations 
from  an  infinity  of  feasible  solutions.  The  condition  for  more  than  one  u- 
nique  solution  to  exist  is  that  the  number  of  unknowns  exceed  the  number  of 
equations.  It  should  be  pointed  out  that  whenever  an  inequality  exists,  an 
additional  unknown  is  introduced  which  converts  the  inequality  to  an  equality. 
This  unknown  is  called  a  "slack  variable"  since  it  is  introduced  to  take  up 
the  slack  (the  extent^  of  the  inequality)  between  both  sides  of  the  equation. 

Once  it  is  established  that  a  great  number  of  possible  solutions  exist, 
an  "objective"  linear  function  of  the  unknown  variables  (raw  materials)  must 
be  set  op  to  enable  the  L.  P.  technique  to  evaluate  the  many  different  possible 
solutions.  Since  the  objective  in  this  problem  is  the  minimization  of  cost, 
a  linear  expression  is  contracted:. 


E  I.  c.  *  Minimum 
i  1  1 


*  An  inequation  represents  an  inequality. 
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.  th 

where  ci  is  the  cost  per  pound  of  the  i  raw  material. 

With  the  problem  formulated*  the  L.  P.  computational  procedure  *  is  used 
to  extract  a  solution*  The  procedure  is  iterative*  requiring  a  systematic 
introduction  of  variables.  As  much  of  each  chosen  variable  is  brought  into 
solution  as  the  restrictive  equations  will  allow*  while  amounts  of  other  vari¬ 
ables  already  in  solution  are  adjusted.  The  method  used  to  choose  the  vari¬ 
able  being  introduced  at  each  step  causes  the  results  to  converge  toward  a 
least-cost  solution. 

THE  SOLUTION  OF  A  TYPICAL  PROBLEM.  A  steel  manufacturer  is  producing 
SAE  310  stainless  steel  with  the  following  charged  composition  (furnace 
losses  are  added  to  the  required  composition*  to  give  the  charged  composi¬ 
tion): 


C  _Mn  Si  Cr  Ni_  P  S 

Percentage t  .25“max  2.00  .75?  2U^26  19-22  .03  max  .03"max 


He  carries  in  inventory  the  materials  shown  in  Table  I*  whose  compositions 
and  costs  are  listed.  To  meet  the  required  end  composition*  and  with  the 
stipulation  that  generated  scrap  can  contribute  no  more  than  30%  of  the  total 
charged  weight*  a  melter's  calculation  gives  the  prescription  per  1000  lb. 
heat  shown  in  the  right  hand  column  of  Table  I*  at  a  total  material  cost  oer 
1000  lbs.  of  $2145.55. 

The  manufacturer  wishes  to  know  if  he  can  reduce  the  average  cost  per 
heat  by  carrying  a  different  set  of  raw  materials  in  stock.  He  studies  the 
market,  and  lists  alternate  materials  to  the  ones  being  used.  From  this 
point*  knowing  the  compositions  and  costs  of  the  alternates,  the  least- 
expensive  combination  of  inputs  that  will  meet  all  the  requirements  can  be 
found  by  L.  P. 


*  Several  algorithms  have  been  developed  to  solve  L.  P.  problems.  The 
most  widely  used  of  these  is  the  simplex  technique*  and  the  interested 
reader  may  learn  the  theoretical  and  computational  details  of  this 
method  from  the  literature.  h,5*o 
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TABLE  I 


COMPOSITION,  % 


Si  Cr 


LBS,  .REQUIRED 
COST  IN  PRESCRIPTION 
Ni  |/ib,  PER  1000  LBS, 


NO.  CORRE¬ 
SPONDING  TO 
MATERIALS 
LISTED  IN 
TABLE  II 

MATERIALS 
STOCKED  IN 
INVENTORY 

2 

FeMn 

5 

FeSi 

9 

FeCr 

11 

Ni 

111 

Pig 

17 

Generated 

Scrap 

18 

Stock 

2.0 

82.5 

1.5 

- 

.15 

- 

.50 

- 

.Oli 

- 

1.5 

70.0 

.1 

CU 

- 

- 

1|.1|2 

.13 

.7 

- 

.22 

1.8 

.65 

25.0 

1A 

CM 

• 

.85 

.30 

.50 

- 

.25 

22.3 

- 

.20 

23  .li 

•ill 

277.7 

99.9 

.7li 

139.1 

- 

.031* 

30.1 

20.0 

.0 

300.0 

.50 

.085 

207. li 

L*  $21:5.55 

Total  cost  of  1000  lb,  prescription 


This  was  done  for  a  total  of  19  materials  (which  are  listed  in  Table  II), 
including  such  alloy-rich  metals  as  silico-manganese  and  ferro-chrome-silicon, 
two  types  of  bar  stock,  and  several  grades  each  of  Fetej,FeSi,  Cr,  FeCr,  Ni* 

The  11  equations  (evaluating  25  unknowns)  which  are  used  to  solve  the  problem 
are  listed  in  Figure. 1,  The  equations  are  analogous  to  those  symbolically 
represented  earlier  with  a  few  additional  refinements:  each  equation  in¬ 
cludes  at  least  one  slack  variable  with  which  is  associated  a  high  cost  (M) 
in  the  objective  function.  These  high-cost  slacks  are  inserted  for  compu¬ 
tational  purposes  only,  and  will  be  driven  out  of  solution  during  computational 
purposes  only,  and  will  be  driven  out  of  solution  during  computation,  hence 
the  name  "artificial”  slack  variables.  Equations  a,  c,  d,  f,  g,  h,  and  i 
contain  additional  real  slack  variables,  because  these  equations  are  in  fact 
inequalities.  The  cost  associated  with  real  slacks  is  zero,  and  ergo  these 
slacks  do  not  appear  in  the  objective  function. 

There  are  two  equations  written  for  the  chrome  balance,  and  two  for  the 
nickel  balance.  This  is  because  requirements  for  these  two  elements  allow 
a  range  of  possible  composition  values.  For  each  of  these  elements,  one 
equation  represents  the  upper  limit  (content  to  be  less  than  or  equal  to  x), 
and  the  other  represents  the  lower  limit  (content  to  be  greater  than  or 
equal  to  y). 
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TABLE  II 

LIST  OF  THE  19  MATERIALS  WHICH  WERE  EVALUATED  IN  THE 
DETERMTNATION  OF  BEST  INPUTS  TO  STOCK  IN  INVENTORY. 


COMPOSITION.  g 


AVAILABLE 


MATERIALS 

C 

Mn 

p 

S 

Si 

Cr 

Ni 

COST  d/LB 

1 

FeMn 

.50 

95.0 

.25 

oo 

1.0 

28.0 

2 

FeMn 

2.00 

82.5 

.35 

.05 

1.5 

- 

- 

25.0 

3 

FeMn 

.10 

82.5 

.35 

.05 

1.25 

• 

mm 

38.0 

2* 

FeSi 

.15 

- 

.05 

.01* 

92.5 

— 

mm 

21*.2 

5 

FeSi 

•1$ 

*  ** 

.05 

.02* 

50.0 

- 

_ 

20.0 

6 

FeSi 

.50 

- 

.06 

.02* 

27.5 

mm 

17.0 

7 

FeSi 

1.50 

- 

.15 

.06 

16.0 

mm 

12*.0 

8 

Cr 

.10 

- 

- 

- 

- 

96.0 

cat 

131.0 

9 

FeCr 

.02* 

Mi 

- 

- 

1.5 

70.0 

— 

2*1.0 

10 

FeCr 

1.0 

00 

- 

- 

1.5 

70.0 

37.5 

11 

Ni 

.10 

- 

- 

.02 

• 

_ 

99.0 

7l*.2 

12 

Ni 

.25 

- 

- 

.05 

- 

- 

98.9 

73.0 

13 

Ni 

.75 

- 

- 

.07 

_ 

* 

97.7 

71.6 

111 

Pig 

2*.i*2 

•13 

.019 

.026 

.70 

mm 

mm 

3.2* 

15 

FeCrSi 

.05 

- 

- 

- 

to.5 

37.5 

mm 

33.0 

16 

SiMn 

1.5 

66.5 

- 

mm 

19.0 

12*.5 

17 

Scrap 

.22 

1.8 

.03 

.03 

.65 

25.0 

20.0 

18 

Stock 

.25 

.85 

•02* 

.02* 

.30 

.50 

.50 

8.5 

19 

Stock 

.13 

.50 

.025 

.025 

.30 

3.5 

1.6 

10.5 

A  matrix  of  coefficients  is  constructed,  and  information  from  the  matrix 
is  encoded  in  machine  language  for  the  computer  to  operate  on.  Hand  or  desk 
calculator  computation  could  be  attempted  depending  on  the  size  of  problem* 
and  for  this  a  simplex  tableau  would  be  constructed.  For  the  particular 
problem  under  discussion,,  hand  computation  is  unfeasible. 

The  machine  solution  to  this  problem  is  shown  in  Table  HI  which  cor¬ 
responds  to  Table  I,  but  lists  the  materials  which  L.  P.  says  will,  when 
combined,  satisfy  the  requirements  and  restrictions  at  least  cost.  This 
least  cost  is  $220.88  or  about  $22*  saved  per  1000  lbs.  ($1*8,000  per  1000 
tons).  The  final  cost  is  most  sensitive  to  those  materials  used  in  greatest 
quantities.  So  if  several  other  bar  stocks  were  evaluated  in  the  problem,  a 
lesser-cost  solution  might  well  have  been  found.  In  general,  the  more  mater¬ 
ials  evaluated,  the  more  likely  are  we  to  find  a  lower  cost  solution.  There 
are  two  reservations  to  the  above  statement.  First,  it  is  entirely  possible 
that  certain  materials  can  be  eliminated  from  consideration  by  inspection 
(aside  from  those  materials  that  will  not  even  do  from  technological  conside¬ 
rations)  .  Second,  it  is  assumed  that  no  extra  computational  costs  are  in¬ 
curred  when  more  materials  are  evaluated.  This  assumption  is  a  reasonable 
one,  especially  when  the  computation  is  by  machine. 


FIGURE  I:  EQUATIONS  USED  IN  THE  LINEAR  PROGRAMMING  SOLUTION 
OF  THE  LEAST-COST-MIX  PROBLEM 
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It  is  possible  to  reduce  the  average  material  cost  per  heat  still  further, 
although  probably  not  to  any  great  extent.  The  number  of  materials  utilized 
in  a  prescription  is  limited  in  the  L.  P.  procedure  by  the  number  of  equations, 
or  rows  in  the  matrix  of  coefficients.  The  number  of  rows  equals  material  in¬ 
puts  plus  slack  variables.  If  one  slack  variable  Could  be  driven  oui^  , 
extra  material  input  could  be  brought  in,  which  might  reduce  the  overall  ma¬ 
terial  cost.  '  A  slack  can  generally  be  driven  oiit  by  reducing  a  requirement 
coefficient  by  the  amount  of  the  slack  which  came  out  of  solution,  and  re¬ 
computing.  This  was  done,  and  the  new  materials  to  be  charged  (8  in  all,  as 
against  7  before)  are  listed  in  Table  IV.  The  new  cost  per  1000  lbs.  is 
$219 .21.  If  such  a  meagre  cost  saving  (compared  with  the  $220.88  prescription) 
is  typical,  the  extra  ordering  and  handling  costs  imposed  by  the  extra  in¬ 
put  would  negate  the  use  of  this  last  prescription,  unless  other  considerations 
prevailed. 

TABLE  III 

NO.  C0RRE- 


SP0NDING  TO 
MATERIALS 
LISTED  IN 
TABLE  II 

MATERIALS 

TO  STOCK  IN 
INVENTORY 

_C _ 

COMPOSITION,  % 

Mn  Si  Cr 

Ni 

COST 

$/ttB> 

.28 

LBS.  REQUIRED 
IN  PRESCRIPTS 
PER  1000  LBS. 

1 

FeMn 

*50 

95.0 

1.0 

9.3 

16 

Si-Mn 

1.5 

66.5 

19.0 

- 

- 

ah 

3.6 

10 

FeCr 

1.0 

- 

1.5 

70.0 

- 

.375 

6ha 

9 

FeCr 

.oh 

- 

1.5 

70.0 

- 

.ill 

169.3 

n 

NL 

.1 

- 

. 

- 

99.5 

.7U 

129.0 

17 

Generated 

Scrap 

.22 

1.8 

.65 

25.0 

20.0 

t>  mm 

300.0 

18 

Stock 

.25 

.85 

.30 

.50 

.50 

.085 

32iu7 

Total  cost  of  1000  lb.  prescriptions2  $220.88 


It  is  a  fact  that  there  almost  always  exist  materials  which  are  substi¬ 
tutes  for  others.  The  L.  P.  method  tests  the  economic  suitability  of  using 
such  substitutes.  For  example,  in  the  last  prescription  a  more  expensive 
bar  stock  (No.  19)  was  chosen,  which  because  of  its  rich  alloy  content,  less¬ 
ened  the  amounts  prescribed  of  certain  ferroalloys.  Thus,  the  final  cost 
was  still  comparable  (in  fact  less,  because  of  the  additional  input  allowed) 
despite  the  use  of  3 ill  lbs.  of  an  "expensive”  material.  It  often  happens 
that  utilization  of  the  expensive  input  results  in  a  cheaper  overall  prescrip¬ 
tion. 


All  L.  P.  computations  were  accomplished  on  an  I.B.M.  6*>0.  A  linear 
programming  earned  program  was  available,  which  made  programming  of  the  com¬ 
puter  unnecesssary.  L.  P.  routines  have  been  written  for  several  of  the  avail¬ 
able  medium  and  large  scale  digital  computers,  and  if  the  foundry  does  not 
own  or  share  such  a  machine,  the  service  bureaus  of  the  larger  computer  manu¬ 
facturers  will  make  them  available  for  a  fee.  Once  again,  an  economic  de¬ 
cision  must  be  made  before  embarking  on  such  a  program.  However,  since  for 
a  machine  comparable  to  the  I.B.M.  6j>0  the  calculation  lasts  but  a  few  minutes, 
and  will  be  required  several  times  a  year  at  most,  the  computational  costs 
are  likely  to  be  reasonable. 
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TABLE  IV 


NO.  CORRE¬ 
SPONDING  TO 
MATERIALS 
LISTED  IN 
TABLE  II 

MATERIALS 

TO  STOCK  IN 
INVENTORY 

C 

COMPOSITION,  % 

Ifa  Si  Cr 

Ni 

COST 

$/LB. 

LBS.  REQUIRED 

IN  PRESCRIPTION 
PEE  1000  LBS. 

1 

FeMn 

.50 

95.0  1.0 

.28 

9.8 

16 

SiMn 

1.5 

66.5  19.0 

.  - 

.11* 

U.5 

10 

FeCr 

1.0 

-  1.5 

70.0 

- 

.375 

121.7 

9 

FeCr 

.ol* 

-  1.5 

70.0 

- 

*1*1 

96.9 

13 

NL 

.75 

- 

- 

97.7 

.72 

7.1 

.  11 

KL 

.1 

- 

- 

99.5 

.71* 

118.2 

17 

Generated 

Scrap 

.22 

1.8  *65 

25.0 

20.0 

300.0 

19 

Stock 

.13 

.50  .30 

3.5 

1.6 

.105 

31*1.7 

Total  cost  of  1000  lbs.  prescription: 

$219.21 

CONCLUSIONS.  Those  foundries  which  have  a  daily  choice  of  charge  inputs 
and  a  computer  on  hand  might  give  some  thought  to  the  calculation  of  charges 
on  a  day-by-day  basis.  Even  without  a  computer,  there  are  successful  manual 
techniques  which  have  been  developed,  usually  incorporating  simplifications 
to  ease  the  computational  burden.  Descriptions  of  these  techniques  can  be 
found  in  the  literature.0 

The  L.  P.  application  described  in  this  paper  allows  a  foundry  to 
a  choice  of  inputs  to  stock  in  inventory.  The  actual  day-by-day  charges 
will  generally  be  calculated  separately,  unless  day-by-day  conditions  are 
identical  to  those  incorporated  in  the  L.  P.  model. ft  For  foundries  using 
L»  P.  to  calculate  the  actual  charges,  D.  E»  Debeau  points  out  a  limitation 
in  the  use  of  the  technique:  the  composition  of  each  charge  input  must  be 
known  within  narrow  limits.  In  those  foundries  where  non-uniform  purchased 
scrap  is  a  major  ingredient,  L.  P.  is  likely  to  prescribe  a  substantial 
amount  of  this  low-cost  scrap,  which  because  of  the  variability  of  its  com¬ 
position  will  result  in  a  relatively  high  number  of  Off -heats. 

A  foundry  may  wish  to  know  the  best  materials  to  stock  in  inventory 
for  a  specific  production  schedule,  or  to  meet  a  fluctuating  demand.  In 
the  first  instance,  the  required  chemical  analyses  for  the  individual  com¬ 
pany  products  can  be  pooled  together,  and  the  optimum  raw-material  w>1t  to 
be  stocked  may  then  be  calculated  by  L.  P.  The  foundry  may  wish  to  recom¬ 
pute  the  optimum  raw-material  mix  whenever  there  is  a  cost-change  in  one  of 
the  inputs,  when  there  is  a  product  requirements  change,  when  new  materials 
become  available,  or  periodically.  If  the  foundry  employs  economic  lot-size 
formuli  for  the  determination  of  stock  quantities,  a  new  prescription  calling 
for  substitute  materials  could  not  be  put  into  effect  until  old  materials  are 
utilized.  Therefore,  the  foundry  might  wish  to  standardize  the  inventory 
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cycle-*-  for  all  charge  input  materials,  and  calculate  the  optimum  charge  by 
L.  P.  at  each  cycle.  **  Whether  or  not  such  a  standard  cycle  is  used,  de¬ 
pends  on  a  cost  analysis  which  needs  to  be  undertaken.  In  general,  the  total 
variable  inventory  cost  for  all  charge  inputs  using  the  standard  inventory 
cycle  time,  -will  exceed  the  total  cost  when  each  input  is  stored  at  its  own 
optimum  level.  Before  adopting  a  change  in  the  inventory  system,  this  extra 
expense  must  be  compared  with  the  estimated  savings  from  Linear  Programming 
prescriptions,  and  the  convenience  of  ordering  all  charge  materials  together. 

Since  each  application  is  individual,  it  remains  for  each  foundry  to  de¬ 
termine  If  such  methods  as  were  described  in  this  paper  could  be  incorporated 
(inevitably  with  alterations)  into  their  system  to  advantage.  The  systems 
analysis  should  be  done  carefully,  lest  some  significant  hidden  cost  or  bene¬ 
fit  be  overlooked. 
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AN  OPTIMAL  METHOD  FOR 
PRODUCT  DEVELOPMENT* 

Sidney  Sobelman 

Picatinny  Arsenal,  Dover,  N.  J. 

In  recent  years,  there  has  been  a  deluge  of  criticisms,  suggestions, 
and  panaceas  for  the  conduct  of  "research  and  development"  and  "product 
development."'  The  newspapers,  scientific  and  engineering  literature,  con¬ 
sulting  management  house  organs,  and  textbook  publishers  have  covered  the 
many  facets  without  analysis  of  the  whole  operation.  Relatively  new  tech¬ 
nical  societies,  like  the  Operations  Research  Society  of  America,  Society 
for  Advancement  of  Management,  and  The  Institute  for  Management  Sciences, 
are  concerned  with  decision-making  in  general  management  with  only  sporadic 
tackling  of  the  management  of  the  R&D  process. 

The  problem  is  essentially  to  develop  a  product,  desired  by  a  customer, 
by  a  date  dictated  by  readiness  for  product  utility  and  by  the  greatest  con¬ 
servation  of  resources  (men  and  money).  There  should  be  a  methodology  which 
can  be  applied  to  any  product,  military  or  commercial,  which  will  assure  the 
optimal  use  of  manpower  (claimed  currently  to  be  in  short  supply)  and  dollars 
(also  a  medium  ever  to  be  conserved)  so  that  a  product  can  be  put  into  use 
at  a  time  commensurate'  with  maximum  pay-off. 

Try  as  one  may  to  separate  the  time  factor  from  allocation  of  manpower 
resources  and  dollars  and  the  reaping  of  profits  or  pay-off,  the  separation 
cannot  be  made— as  will  be  demonstrated  throughout  this  exposition.  Tech¬ 
nological  decay  and  environmental  decay  and  the  present  worth  of  future 
dollars  further  dictate  the  consideration  of  the  time  factor  in  evaluating 
costs  and  pay-off.  To  resign  ourselves  to  haphazard,  random,  or  unscheduled 
conduct  of  programs  (i.e.,  projects  with  loose  or  no  completion  dates)  is 
to  disregard  the  importance  of  timing  and  consequently  of  technological  de¬ 
cay  of  a  current  product  and  useful  life  of  a  future  product. 

There  is  a  serious  difficulty  in  getting  across  the  concept  that 
developments  of  military  and  of  commercial  products  do  not  differ  in  any 
way.  There  is  general  acceptance  today  of  the  "fact"  that  the  former  does 
not  involve  a  profit  motive  to  the  organization  whereas  the  latter  does. 
Suppose  we  now  change  the  phrase  "profit  motive"  to  "pay-off  value"  which 
is  broader  in  meaning  and  certainly  encompasses  the  concept  of  profit  taking. 
This  change  to  the  use  of  broader  (and  more  basic)  terminology  must  be 
accepted  by  even  the  most  reticent  but  still  leaves  us  with  the  difficulty 
of  relating  "pay-off":  for  both  military  and  commercial  products  in  the  same 
terms,  as,  for  example,  in  terms  of  dollars.  Even  this  difficulty  can  be 
overcome  logically  and  with  utmost  meaning  and  utility. 


*  Based  on  a  report  "A  MODERN  DYNAMIC  APPROACH  TO  PRODUCT  DEVELOPMENT", 
Dec  1958,  made  under  the  auspices  of  a  Secretary  of  the  Army's 
Research  and  Study  Fellowship,  and  to  be  available  from  Office  of 
Technical  Services,  Dept,  of  Commerce,  in  July  1959. 
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Returning  to  the  topic  of  importance  of  continual  and  burgeoning 
development  of  new  products,  it  is  noted  that  the  impact  of  ’’research  and 
development”  in  a  technology  changing  at  an  increased  rate  has  taken  on 
national  and  local  importance,  often  with  ominous  overtones.  Perhaps  the 
most  recent  expressions  of  this  problem  are  the  paper  by  Ellis  A.  Johnson, 
"The  Crisis  in  Science  and  Technology  and  its  Effect  on  Military  Develop¬ 
ment”  (Ref.  1)  and  the  paper  by  J.  Sterling  Livingston,  "Decision  Making 
on  Weapons  Development"  (Ref.  2).  One  critical  point  in  this  picture  is 
the  fact  that  time  periods  of  7-8  years  up  to  10  years  for  the  development 
of  a  product  are  unsatisfactory  when  its  useful  life  might  only  be  5  years 
(e.g.,  for  guided  missiles).  Indeed,  a  needed  product  might  be  obsolescent 
by  the  time  it  is  designed,  developed,  and  ready  for  production  or  issue 
to  the  user  or  customer. 

The  problem  restated  is  that  for  any  product  in  a  rapidly  changing  or 
competitive  technology  the  history  of  an  item  can  be  longer  than  desired 
and  that  the  useful  life  of  the  product  may  not  be  long  enough.  Besides 
time,  and  the  long  and  short  of  it,  the  product  to  be  developed  must  have 
a  "pay-off"  sufficiently  high  so  that  we  do  not  lose,  along  with  competi¬ 
tive  position,  the  costs  and  resources  expended  in  product  development. 

The  components  of  the  problem  restated  are  as  follows: 

1.  Pay-off  or  net  profit  (p) 

2.  Useful  life  or  market  life  (time  period  for  profit  taking)  (T) 

3.  Development  costs  (c) 

L.  Development  time  (t) 

The  decision  to  acquire  or  start  the  development  of  a  new  product, 
or  improve  a  current  product,  is  so  important  that  it  can  only  be  con¬ 
ceived  of  as  being  the  responsibility  of  top  management.  Except  when  top 
management  wishes  to  pay  for  prestige,  for  denial  of  a  market  to  a  com¬ 
petitor,  or  for  other  similar  intangibles,  the  decision  must  be  based  on 
a  quantitative  measure  of  pay-off.  In  order  to  have  high  levels  of  pre¬ 
cision  and  accuracy,  the  estimates  going  into  computation  of  pay-off  should 
be  prepared  by  the  specializing  departments  who  will  later  be  expected  to 
execute  the  development,  production,  servicing,  and  other,  programs. 

A  viewpoint  different  from  that  often  expressed  in  discussions  or 
other  papers  is  also  necessary  concerning  the  terms  "state  of  the  art," 
"degree  of  newness,"  "risk  or  probability  of  successful  conclusion."  It 
is  opinionated  that  such  terms  have  seriously  retarded  progressive  and 
dynamic  thinking  and  solution  of  the  evaluation  and  selection  problem. 

Some  contemplated  products  are  recognized  as  having  very  little  prior 
experience  to  go  by  in  design,  production,  or  use.  The  "state  of  the  art" 
is  considered  to  be  low  and  the  "degree  of  nearess"  is  high.  With  such  an 
item  it  is  also  likely  that  the  "probability  of  successful  conclusion"  is 
considered  very  low.  What  do  we  really  mean  by  such  terminology?  Do  we 
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not  really  mean  that  there  is  not  enough  knowledge ,  facts,  and  experience 
for  us  to  feel  really  confident  about  the  future?  Well,  similar  situations 
exist  to  a  lesser  degree  with  all  but  the  very  simplest  developments.  The 
crux  of  the  problem  is  that  all  unknown  facts  and  experiences  «a.n  be  obtained 
at  some  cost  from  applied  research  and  pilot  plant  lots,  thorough  market 
analysis,  and  thorough  testing*  We  are  stating  that  any  development  can  be 
successful— —if  we  are  willing  to  pay  enough  in  dollars  and  manpower* 

There  is  now  a  true  meaning  to  the  risk  of  being  tin  successful .  There 
is  a  very  definite  risk  of  not  successfully  concluding  the  development  and 
marketing  of  a  product  if  we  have  in  any  way  stinted  on  funds  and  personnel 
necessary  to  have  a  thorough  market  analysis  or  feasibility  study,  to  acquire 
missing  data  and  understanding  of  processes  involved,  to  test  out  all  phases, 
or  for  the  design  and  development  itself.  The  risk  of  success  can  then  be 
recognized  as  the  ratio  of  the  funds  actually  supplied  to  the  total  funds 
necessary  for  successful  completion  of  the  product.  There  can  also  be  a 
risk  of  failure  in  meeting  a  completion  date,  no  matter  how  much  funding 
and  personnel  are  furnished;  but  on  the  other  hand  there  are  examples  of 
very  difficult  complete  developments  and  production  of  very  new  products 
in  as  short  a  time  as  lb  months  to  2  years.  ~" 

With  the  statement  of  a  "reasonable”  completion  date  there  can  be  no 
such  claim  of  a  risk  of  failure— it  is  only  a  question  of  how  much  do  you 
want  to  pay  to  assure  a  successful  development. 

Before  considering  a  model  for  the  system,  which  model  is  to  be 
optimized  for  maximum  pay-off  to  the  organization,  we  must  discuss  how 
pay-off  or  profit  for  a  single  product  in  the  system  can  be  determined. 

In  -the  search  for  a  meaningful,  absolute  measure  of  the  pay-off  or 
net  profit  of  a  proposed  product,  where  maximizing  the  sum  of  all  the  net 
profits  is  the  prime  objective  of  the  firm,  the  simplest  most  direct  rela¬ 
tion  that  can  be  found  is  that  of  subtracting  all  the  costs  chargeable  to 
the  product  from  the  gross  income.  But  this  is  not  as  useful  as  separation 
of  the  development  costs  from  other  costs  since  we  are  concerned  with  the 
magnitude  of  development  costs  and  since  these  costs  occur  in  a  different 
time  period  from  other  product  costs*  Further,  we  have  seen  that  time  has 
an  economic  value  and  that  market  life  is  distinct  from  developraentTif e . 

We  thus  progress  from  equation  1  to  2  to  3,  as  follows: 

Eq.  1  z  =  P 

Eq.  2  z  =  P  -  C 

Eq.  3  z  =  pT  -  ct 

where  z  =  product  value,  P  =  total  net  profit,  C  =  total  development  cost, 
p  -  average  net  profit/year,  T  =  market  or  useful  life  in  years,  c  -  average 
development  cost/year,  and  t  =  years  of  development. 

The  definitions  of  the  equation  3  factors  are  stated  as  they  are  for 
ease  of  computation,  fitting  in  with  forecasting  and  scheduling,  and  with 
budgeting  procedures*  For  example,  the  basis  of  a  year  is  used  in  most 
budgeting  and  discounting  procedures*  The  profit  and  cost  picture  per 
year  is  usually  desired,  as  well  as  the  total  profit.  Further,  it  is 
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convenient  and  possible  to  express  annual  costs  and  profits  in  terns  of  the 
average  or  arithmetic  mean,  even  though  there  are  variations  between  years j 
we  thus  are  considering  the  same  total,  i •  e  • ,  the  area  under  the  curve, 
whether  yearly  values  are  constant  or  variable,  but  using  a  square  distri¬ 
bution  instead  of  a  curvilinear  one. 

Equation  3  is  basic  and  it  is  complete  as  an  expression  of  pay-off  and 
net  profit  for  one  product.  Every  product  characteristic  and  its  magnitude 
contributes  to  increasing  one  of  the  four  parameters  of  the  equation.  In 
the  final  analysis,  though,  it  is  the  design  of  the  product  that  establishes 
the  rough  location  of  costs  and  it  is  the  degree  of  efficiency  of  allocation 
of  manpower,  time,  facilities,  and  materials  that  pinpoints  the  exact  cost. 

It  can  be  seen  that  in  early  evaluation  of  a  product's  value,  where  the  de¬ 
sign  or  designs  are  only  generally  conceived,  only  rough  locations  of  the 
pay-off  are  possible.  As  progress  is  made  in  development  and  a  design  becomes 
more  firm,  the  optimization  of  later  use  of  facilities,  manpower,  and  ma¬ 
terials  can  be  studied  leading  to  closer  and  closer  pin-pointing  of  costs 
as  well  as  pay-off.  The  formula  can,  and  should  be,  used  for  periodic  re- 
evaluations  of  the  product  value. 

The  economic  value  of  time  can  be  accounted  for  by  discounting  future 
dollars  for  their  present  value.  Since  costs  and  profits  can  be  stretched 
out  in  time  by  years,  discounting  can  be  applied  by  multiplying  dollars  by 
(1  +  i)“^  or  its  approximation,  e-it.  However,  two  areas  of  time  are  still 
not  accounted  for,  one  being  the  desire  to  maximize  both  "short-range"  and 
"long-range"  profits  and  the  other  being  the  problem  of  evaluating  pay-off 
whether  in  a  peace-time  or  war-time  economy.  If  our  formula  (Equation  3) 
can  be  modified  to  take  care  of  these  two  "time"  areas  it  has  then  become 
even  more  basic  and  useful.  An  attempt  to  do  this  follows. 

Getting  "short-range"  profits  or  benefits  is  not  necessarily  incom¬ 
patible  with  maximizing  "long-range"  profits.  From  the  short-range  view¬ 
point  we  want  to  recognize  and  reward  those  projects  which  have  shorter  than 
average  development  times  so  that  profit-taking  can  start  earlier  than  it 
would  have  if  we  had  a  project  of  average  development  time.  (For  any  class 
or  type  of  products,  managers  and  historians  will  recognize  the  existence 
of  an  average  development  time  expected  for  that  class  or  type,  e.g., 
recent  figures  indicate  10  years  development  for  a  guided  missile,  $  years 
for  a  new  electrical  appliance,  3  years  for  a  major  modification  of  an 
automobile.)  We  must  also  recognize  that  development  time  can  be  shortened 
or  lengthened  almost  at  will  by  the  demand  of  the  customer,  user,  the  direc¬ 
tor  of  the  New  Products  Department,  or  the  Sales  or  Logistics  Department; 
changing  of  the  expected  delivery  date  would  also  be  expected  to  affect 
the  cost  of  development. 

The  relation  then  is  that  a  development  time,  t,  equal  to  -that  of  the 
average,  t,  does  not  affect  ou*j>rofit  picture,  but  if  t<t  there  is  an 
ancrease  in  total  net  profit,  pT,  and  if  t  >  t_there  is  a  loss  of  profit. 

Figure  1A  illustrates  this  relationship  when  t  »  3  years.  Equation  4  also 
represents  this  relationship  (assumed  linear) . 

A(pT)  =  (-pT/t)t  +  pT  =  pT  (1  -  t/t) 


Eq.  4 
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Note  that  A(pT)  =  0  (no  change  in  pT,  total  net  profit)  when  t  =  t.  Also, 
A(pT)  =  -pT  (  the  loss  of  net  profit  of  one  whole  average  project)  when  de¬ 
velopment  time  is  twice  as  long  as  that  of  the  average  project,  i„e. ,  when 
t  =  2 1„ 

From  the  long-range  viewpoint,  we  would  prefer  products  that  have  a 
market  life  longer  than  that  of  an  average  product  of  the  same  type  or  class 
In  fact,  if  a  product  has  a  useful,  profit-making,  or  market  life  that  is 
twice  as  long  as  the  average  it  has  in  fact  saved  us  the  necessity  and  cost 
of  one  whole  development  project,,  Thus,  if  a  product  will  last  6  years  in¬ 
stead  of  the  average  of  3  years  which  is  known  to  exist  for  similar  products 
there  is  no  need  for  a  development  program  to  provide  a  product  to  cover  the 
profit-taking  of  the  last  3  years  of  the  6.  Figure  IB  illustrates  this 
relationship  and  is  represented  by  Equation  5° 

Eq„  5  A(ct)  =  (ct/f)T  -  ct  =  ct  [(t/t)  -  l] 

Adding  the  A(pT)  and  A(ct)  of  Equations  (4)  and  (5)  to  Equation  (3),  we 
gets 

Eq.  6  z  =  pT  +  pT  (l  -  t/t)  -  ct  +  ct  [(T/T)  -l] 
and  simplifying  by  rearrangement  of  terms: 

Eq.  7  z  «  p  [t  +  T(l  -  t/t)]  -  c[t  +  t  (1  -  T/f)] 

If  T*  =  [t  +  T  (1  -  t/t)| 
and  t*  =  [t  +  t  (1  -  t/t3 
then,  when  T  /  T  and  t/t: 

Eq«  &  z  =  pT*  -  ct* 

When  evaluating  programs  periodically „  using  our  recommended  formula, 
after  development  has  been  started,  the  value  for  »t"  is  that  of  the  re- 
maining  years  required  for  completion .  Any  previous  year's  development 
costs  are  thus  excluded  from  the  evaluation  of  future  product  worth.  This 
is  considered  sound  in  that  we  are  always  considering  the  present  worth 
of  the  product's  future,  regardless  of  the  past.  To  consider  the  inclusion 
of  past  years  and  their  development  costs  is  in  the  direction  of  reduction 
of  present  worth  and  may  unfairly  jeopardize  a  program  in  competition  with 
newly  conceived  products.  Retrospection  will  show  that  this  method  ac¬ 
tually  protects  our  past  expenditures  already  invested  in  the  development 
program— but  without  the  need  for  considering  how  much  time  or  costs  were 
spent  |  our  only  interest  is  the  present  worth  of  future  pay-off. 

Concerning  our  previously  stated  problem  of  application  to  both  peace¬ 
time  and  war-time  situations,  the  following  is  quoted  from  A.S.  Locke 
(Ref  3)  ‘  "The  unit  by  which  cost  is  measured  is  different  in  a  peacetime 
economy  from  that  in  a  wartime  economy.  When  a  nation  is  at  war,  the 
determining  unit  is  that  of  time.  If  a  known  threat  exists,  a  counter  wea¬ 
pon  must  be  put  into  action  against  it  in  the  shortest  possible  time.  The 
cost  in  dollars  becomes  unimportant;  the  cost  in  time  becomes  all  important. 
In  peacetime,  the  cost  of  the  weapon  program  in  units  of  dollars  becomes 
the  governing  factor  between  competitive  weapon  systems." 


A(pT)s(-pT/t)t  +  pT  =  p?(l-t/t). 

A.  RELATION  OF  DEVELOPMENT  TIME  AND 
ANNUAL  PROFIT. 
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A(ct)=  (ct  /  T  )  T  -  ct  =  ct  [(T/T)  -l] 

B.  RELATION  OF  MARKET  LIFE  AND  ANNUAL 
DEVELOPMENT  COST. 


figure 
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By  properly  relating  pay-off,  costs,  and  time,  as  in  Equation  8,  we  have 
given  a  very  short  time  of  development,  most  desirable  in  wartime  (and  not 
really  any  less,  in  peacetime),  such  effect  in  increasing  profit  (by  the  low 
value  of  t/t  in  T*)  that  the  "cost  in  time"  overshadows  the  development  costs <> 
This  writer  does  not  therefore  consider  that  wartime  or  peacetime  causes  a 

change  in  how  we  measure  costs  but  that  we  forcibly  put  into  effect  in  wartime 

measures,  such  as  short  development  times,  that  would  also  have  increased 
pay-off  if  it  had  been  peacetime. 

To  illustrate  the  basic  nature  of  our  development  of  Equation  8  as  a 
measure  of  pay-off  or  importance  of  a  product,  we  refer  to  Equation  6  which 
is  rewritten  as  follows; 

Eq,  9  z  -  pT  +  pT  (1  -  t/t)  -  ct  -  ct  (l  -  T/f) 

For  small  values  of  t/t  and  T/f,  we  can  write  (l  -  t /t)  as  e_t/^  and 

(1  -  t/t)  as  e  o  Thus,  we  get; 

Eq,  10  z  =  pT  +  pT  °  e  —  ct,  —  ct  °  e 

By  comparison  with  a  physical  system  for  high  polymers  as  given  by 
T„  Alfrey  (Ref  4),  where  S  =  Sq  e-®/** is  the  experimental  timescale  or 

actual  time  of  application  of  a  stress,  S,  whose  initial  value  is  S  and 
r  is  "thought  of  as  an  internal  timescale,  characteristic  of  the  reJaxa- 
tion  of  the  material,"  Thus,  by  comparison  our  t  and  T  are  averages  charac¬ 
teristic  of  the  class  or  type  of  product  and  our  t  and  T  are  the  experimental 
or  estimated  development  times  and  product  lifetimes,  (One  should  not  attempt 
to  substitute  actual  values  in  Equation  10  without  a  correction  factor.) 

A  sensitivity  analysis  for  the  errors  of  estimating  should  be  made  for 
Equation  8.  The  method  for  doing  this,  and  the  results  for  one  example, 
are  presented  below. 

The  parameters  in  Equation  8  are  engineering  and  sales  engineering  or 
market  analysis  estimates.  The  chances  are  that  the  profit  and  cost  errors 
occur  at  random  and  in  estimating  may  as  often  cancel  each  other  as  augment 
each  other.  Further,  estimating  errors  may  tend  to  cancel  each  other  be¬ 
tween  projects  as  well  as  within  projects  for  any  one  program.  However,  an 
estimate  of  the  maximum  errors  and  the  expected  dispersion  of  values  due  to 
estimating  errors  will  point  out  the  degree  of  accuracy  needed. 

Using  z  =  pT  -  ct,  when  T  =  f ,  t  =  t,  we  find  that; 

Eq.  11  z  +  Az  =  (p  +  Ap)  (T  +  AT)  -  (c  +  Ac)  (t  +  At) 

and  by  combining  terms,  by  making  all  terms  positive  for  maximum  deviation, 
by  dropping  Ap  °  AT  and  Ac  •  At  terms  as  very  small,  and  by  converting  to 
"Ax/x"  fraction  equivalents  of  percentage  error,  we  get; 

Eq.  12  Max.  Az  V  pT  ♦  ££)  +  ct  (^  +  ^£) 

Let  p  =  $450, 000/year,  and  T  =  3  years,  pT  =  $1,350,000?  and  c  =  $150,000/ 
year  and  t  *  3  years,  then  ct  »  $450,000. 
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The  error  in  estimating  years  of  development  and  useful  or  market  years 
is  expected  to  be  small,  particularly  if  estimates  and  planning  are  done 
carefully,  and  let  us  say  that  these  are  not  more  than  5%  in  error.  For  de¬ 
velopment  costs,  it  is  believed  that  these  can  be  estimated  reasonably  with 
a  10%error  provided  optimism  and  enthusiasm  does  not  make  one  forget  the 
usual  and  expected  need  to  redesign  and  retest  one  or  more  times.  More 
serious  errors  are  expected  in  estimating  the  expected  yearly  profit  and 
this  perhaps  would  be  a  20% error.  Thus: 

Ap/p  =  0.2 

Ac/c  =  0.1 

At/t  and  AT/T  =0.05 

Substituting  values  in  Equation  12 ,  we  get : 

Max.  Az  =  1,350,000  (0.25)  +  450,000  (0.15) 

=  337,500  +  67,500 

=  405,000 

405,000/(1,350,000  +  450,000)  =  0.225,  or  22.5  for  maximum  error  in  z, 
the  product  value. 

Since  several  distributions  are  added,  we  can  expect  the  distribution 
for  the  error  in  z  to  be  normally  distributed  and  0.225  corresponds  to  the 
3  o  limit.  The  a  =  0.075  and  the  Probable  Error  is  0.6745  (O.O75)  or  0.051. 
Thus  we  can  expect  that  in  about  68% of  our  estimates  z  may  have  an  error  of 
7.5%or  less  and  half  of  our  estimates  will  have  errors  of  5.1% or  less. 

There  is  another  important  concept,  useful  in  management  control,  that 
can  be  gleaned  from  our  basic  relationship,  This  concept  concerns  "maturity" 
of  a  class  or  type  of  products.  Suppose  we  rewrite  Equation  7  as  follows: 

Eq.  13  z  =  pT  +  pT  -  pt  (T/t)  -  ct  -  ct  +  cT/(f/t) 

If  k  =  (T/t) ,  then  the  larger  this  ratio  of  useful  life  to  development 
the  greater  the  product  value,  z,  for  future  average  projects  because  of  both 
increased  profit  and  decreased  development  cost.  The  factor,  k,  then  is  a 
measure  of  the  "maturity,"  or  position  on  the  growth  curve  so  characteristic 
of  the  learning  process,  aging  of  animal  life,  growth  of  populations,  and 
radioactive  transmutations  (see  Feller,  Ref  5).  For  a  mature  product,  k  is 
high  as  for  hand  grenades  having  an  approximate  useful  life  of  25  years  with 
development  in  the  area  of  5  years,  k  =  20/5  =  4.  A  guided  missile,  on  the 
other  hand,  has  the  expected  life  of  5.0  years  but  takes  10  years  to  develop, 
hence  k  =  0.5.  The  product  class  of  nylon  might  be  expected  to  have  a  low 
value  of  maturity  but  because  of  the  thorough  basic  research  and  the  devel¬ 
opment  by  Carothers  and  his  duPont  staff  it  actually  has  a  high  maturity; 
we  take  the  liberty  of  estimating  that  its  life  will  go  another  10  years, 
hence  a  total  of  30  years  (1939  -  1969)  whereas  development  (excluding  the 
basic  superpolyamide  research)  might  be  of  the  order  of  7  years  (l932  -  1939)  , 
hence  k  =  30/7  =  4.3.  For  the  human  species,  by  way  of  comparison,  devel¬ 
opment  might  be  accepted  by  most  as  18  years  and  useful  life,  from  18  to 
65;  hence,  k  =  47/18  =  2.6. 
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The  thought  now  occurs  that  there  is  cost  associated  with  and  a  pay-off 
resulting  from  an  increase  in  development  time  which  results  in  a  longer  use¬ 
ful  life  for  a  product.  The  useful  life  of  a  product  is  a  direct  result  of 
its  characteristics  and  design  insofar  as  how  thoroughly  it  satisfies  a  mar¬ 
ket  or  customer's  need.  The  thoroughness  tad  development  costs  put  into  the 
development  of  nylon  is  still  paying  off.  The  same  thoroughness  and  high 
costs  were  necessary  for  the  very  existence  of  a  guided  missile  and  gains  in 
reduction  of  development  time  and  costs  can  be  expected  in  the  near  future 
for  any  new  guided  missile  required  by  "market"  demands.  Using  the  analogy 
of  human  development,  a  delay  in  starting  one's  useful  life  at  18,  as  by 
going  to  college,  increases  both  pay-off  during  the  working  life  of  an  indi¬ 
vidual  and  may  even  extend  his  working  years  beyond  65  (if  he  and  his  employer 
so  desire).  Further,  the  realization  of  the  problem  and  the  ability  of  an 
agency  or  person  to  design  a  more  useful  and  longer-lasting  product  together 
with  its  knowledge  of  scheduling  which  leads  to  reduced  development  times 
contributes  to  higher  values  of  k.  Does  this  not  lead  to  a  measure  of  the 
"efficience"  or  "maturity"  of  that  agency  or  person? 


In  order  to  determine  the  maximum  product  value,  z  ,  we  take  the 

max 

partial  differential  of  z  in  Equation  13  with  respect  to  profits  and  sep¬ 
arately  with  respect  to  costs,  and  proceed  as  follows ’ 


Eq.  14  ds 
dp 


=  T  +  T  -  — 


3  =  0;  T  = 


T(t-r  t) 


Similarly  for  cqsts: 

Eq.  15  dz  _  .  t  Tt 
_  -  t  -  t  +  — 

T 


-  0,  T  -  Ht-lH 
t 


From_Equations  14  and  15  we  get  the  expected  results  that  the  optimal 

value  of  t  =  0  and  the  optimal  value  of  T_=  (T/t) .  •  t  =  k  •  t  (i.e.,  when 

t  ■  0).  Using  these  two  values  of  t_and  T  in  Equation  13,  we  get  the  simple 

and  significant  result  that  z  ■  pT.  Thus,  if  the  characteristic  of  a 

max 

product  is  that  it  requires  no  development  time,  that  is,  can  be  created 
instantaneously  like  Adam,  or  a  god  in  Greek  mythology,  or  by  immediate 
purchase,  company  merger,  or  similar  means,  only  then  can  we  get  the  max¬ 
imum  pay-off  or  profit.  In  such  a  case,  the  cost  of  purchase  or  investment 
would  have  to  become  a  cost  against  the  calculation' of  p,  the  annual  net 
profit;  the  discounted  cash-flow  method  analysis  or  similar  investment  anal¬ 
ysis  might  then  be  applicable.  It  is  also  interesting  to  note  that  for  a 
proven  or  well-established  ("standard")  value  of  k  (or  T/t)  for  a  class  of 
products,  we  would  do  best  by  having  T/t,  the  estimated  ratio  of  product 
life  to  development  time  for  a  newly  prpposed  product  under  evaluation,  as 
equal  to  k.  But  we  can  do  better  by  investing  in  the  improvement  of  the  k 
value  for  the  whole  class  of  products  by  advancing  the  "state  of  the  art" 
or  "maturity"  through  applied  research  and/or  improvement  of  the  methods 
of  scheduling  and  allocation  of  development  resources. 
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When  t  /  0,  but  we  still  have  T  =  k°t ,  the  maximum  value  of  z  becomes? 
Eq.  16  z  =  pT  -  ct 
or  rewritten  for  plotting  on  a  graph  as  in  Figure  2: 

Eq.  17  z  m  T  #  £  _  1 
ct  t  c 

where  —  =  k  (slope  of  the  line).  Figure  2  is  plotted  for  the  condition  of 
t 

k  =  1.  We  now  want  to  know  the  value  of  improving  the  "maturity"  of  class 
of  products  from  k  =  1  to  k  »  2,  keeping  the  same  ratio  of  the  annual  average 
profit  to  annual  average  cost,  in  this  case  p/c  =  2.  The  expected  improve¬ 
ment  to  do  this  would  be  expected  to  be  the  reduction  of  one-half  the  cost 
of  any  future  whole  new  product  development  project.  The  higher  the  ratio 
of  profits  to  costs  or  the  higher  we  wish  to  advance  the  "maturity"  of  a 
class  of  products,  the  more  beneficial  does  this  activity  become.  The  cost 
of  improving  "maturity"  is  the  separate  estimate  of  the  necessary  applied 
research  weighed  against  the  value  of  improvement  for  one  or  more  expected 
new  products  in  the  same  class  or  type. 

For  a  commercial  product,  the  use  of  dollars  makes  it  comparatively 
simple  to  compute  the  gross  profit  or  pay-off  and  subtract  from  it  all  the 
costs  to  obtain  the  net  profit  in  dollars.  The  problem  is  now  to  do  the  same 
for  the  pay-off  for  a  military  product,  which  is  often  expressed  in  terms 
of  fire-power,  probability  of  kill,  percentage  improvement  over  a  competi¬ 
tive  existing  product,  effect  on  time  reduction  of  a  major  war,  ability  to 
defeat  a  specific  target,  and  so  forth.  Unless  this  can  be  done,  there 
appears  to  be  little  hope  of  ever  obtaining  an  absolute  measure  of  worth 
for  a  military  product. 

Let  us  attempt  to  establish  some  basic  relations  for  all  military  pro¬ 
ducts,  as  follows: 

1.  The  purpose  of  such  products  is  their  possible  use  in  a  major  w,ar 
and  evaluation  of  net  pay-off  should  be  made  under  such  circumstances. 

This  does  not  preclude  their  use  or  make  them  less  valuable  in  limited 
wars  or  police  actipns.  Once  net  pay-off  is  determined,  it  then  is  the 
value  to  be  used  as  the  "p"  value  in  our  formulas. 

2.  There  is  a  cost  of  getting  closer  physically  to  the  enemy  or  an 
enemy  target  in  order  to  capture  or  destroy  the  same.  This  must  be  recog¬ 
nized  to  relate  the  costs  and  pay-offs  of  weapons  having  the  same  effect¬ 
iveness  but  at  different  ranges. 

3„  All  ntilitary  products  can  be  and  should  be  evaluated  ultimately  in 
terms  of  the  major  objective  of  defense  of  the  nation  and  all  its  resources, 
of  the  military  forces  (actually  a  resource  in  its  broadest  sense),  and  of 
any  segment  of  the  military  forces  down  to  the  individual  soldier.  The 
primary  purpose  of  an  offensive  action  and  of  the  ultimate  need  to  win  is 
the  defense  of  the  nation,  its  way  of  life,  and  what  it  stands  for. 
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THE  DETERMINATION  OF  VALUE  OF 
IMPROVING  PRODUCT  “MATURITY'.' 

Figure  2 


Fourth  Conference  on  Operations  Research 


115 


4*  The  basic  unit  requiring  defense  in  the  war  effort  is  the  individual 
soldier.  As  distasteful  as  it  must  be  to  all,  a  value  in  dollars  must  be 
placed  on  this  soldier  under  combat  conditions  and,  based  on  an  estimate  of 
cost  of  conversion  from  a  civilian  to  a  trained  combatant,  let  us  state  this 
will  be  $8,000.  For  other  units,  such  as  a  factory,  steel  mill,  industrial 
city,  we  will  again  have  to  be  heartless  and  calculating  and  use  a  dollar 
sign  measure  of  its  contribution  to  defense,  let  us  say,  in  terms  of  days 
of  sustained  combat,  for  which  we  can  get  a  cost  measure. 

An  example  will  illustrate  the  possibilities  of  establishing  dollar 
values  of  pay-off.  Hypothetical  values  only  will  be  used,  as  well  as  rounded- 
off  numbers  for  ease  of  computation.  Let  us  take  the  pay-off  for  a  rifle.  If 
a  war  of  atomic  warhead  long-range  ballistic  and  guided  missiles  should  come 
to  pass,  it  is  possible  to  so  devastate  a  country  that  rifles  will  be  of 
little  or  no  value.  However,  with  opposing  nations  having  retaliatory  abil¬ 
ities,  it  is  opinionated  that  it  is  not  likely  that  this  type  of  warfare  will 
be  tried  except  as  a  last  desperate  effort.  So  we  will  then  consider  that 
the  rifle  has  a  purpose.  We  will  consider  that  3  million  semi-automatic 
rifles  will  be  manufactured  and  2  million  will  be  issued  to  troops.  Cost  of 
a  rifle  being  set  at  $100  each,  the  cost  is  3  ^  10^  dollars.  From  our  logis¬ 
tics  people  and  field  commanders ,  let  us  say  that  we  find  that  because  of 
our  infantry  being  equipped  with  rifles,  we  can  save  300,000  GI's  in  a  war 

of  3  years*  duration.  Hence,  the  gross  pay-off  is  (8  x  103) (3  x  10  5) 

Q 

=  2,4  x  10  dollars  against  which  we  have  a  cost  of  production  only,  of 
3  x  10°.  This  represents  a  loss,  or  an  absence  of  a  net  pay-off.  We  are 
accomplishing  the  defense  objective  of  saving  of  our  own  lives  but  not  very 
efficiently.  Suppose,  however,  we  contemplate  the  development  of  an  auto¬ 
matic  design  to  replace  the  semi-automatic  rifle ,  and  there  is  an,estimate 
that  we  can  now  save  380,000  lives.  The  gross  pay-off  is  (8  x  l(r)(3 .8  x  105) 
=  3.04  x  10° ,  with  a  net  profit  of  4  x  10'  dollars  (considering  production 
costs  only).  In  computing  the  total  product  value  by  z  =  pT  -  ct,  let  us 
remember  that  there  is  a  development  cost  to  be  subtracted  from  the  total 
net  pay-off.  The  product  value  might  be  something  like  the  following  (in¬ 
cluding  discounting  for  present  worth  of  future  dollars  to  get  the  discounted 
average  annual  p  and  c) : 

z  =  pT  (1  +  a)  _  ct  (1  +  a)^k,  where  T  =  T  and  t  =  t 

-  (4  x  107)(20)(1.03)”16-  (5  x  105)(6)(1.03)3 
=  (80  x.l07)/l.6l  -  (30  x  105)(l.09) 

=  497  x  106  -  3.27  x  106 
=  4.94  x  108  dollars 

Time  and  space  cannot  be  devoted  to  the  methods  of  computing  lives 
saved,  the  cost  in  terms  of  one  day  less  of  war,  the  value  in  defense  dollars 
of  an  industrial  plant,  and  so  forth,  since  each  example  will  be  an  exercise 
in  itself  with  innumerable  estimates  and  considerations  of  probabilities  which 
the  logistics  and  field  commanders  know  best.  It  can  only  be  indicated  that 
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we  have  a  total  economy  to  defend,  whether  by  defensive  or  offensive  means 
in  varying  degrees.  The  values  of  alternate  approaches  and  products  for  this 
defense  can  be  analyzed  in  terms  of  dollars,  if  we  really  desire  to  do  so,  by 
the  use  of  ingenuity,  imagination,  and  logic.  However,  the  value  of  products, 
commercial  or  military,  in  terms  of  personal  satisfaction,  political  aspira¬ 
tions,  cultural  gains,  protection  of  a  way  of  life,  good  Will,  etc.,  cannot 
be  evaluated,  unless  the  reader  wishes  to  put  a  dollar  value  on  any  of  these 
''intangibles,"  or  on  their  impact  on  an  economy. 


The  preceding  discussed  a  recommended  means  for  determining  the  value 
of  a  product  based  on  estimates  of  annual  net  pay-off,  product  life,  devel¬ 
opment  cost',  and  development  time.  In  no  way  did  we  prove  that  this  will 
give  us  the  optimal  method  for  maximizing  profit  (or,  on  the  other  side  of 
the  coin,  minimizing  costs)  to  the  firm  over  either  or  both  i^s  short-term 
or  long-range  existence,  that  is,  that  the  method  meets  the  basic  objective 
of  the  firm.  So  far  we  have  discussed  how  one  program  is  evaluated;  we  must 
now  consider  selecting  those  programs  out  of  the  set  of  all  possible  pro¬ 
grams  which  will  maximize  profits  over  the  life  of  the  firm. 

Hertz  and  Feeney  (Ref  6)  have  prepared  a  basic  analysis  of  an  organized 
industrial  "operation"  which  is  also  basic  to  this  study.  The  development 
(and  proof)  of  the  recommended  procedure  of  this  study  will  follow  along 
similar  lines  but  with  modifications  primarily  to  include  additional  values 
of  time  itself  and  of  the  separation  of  development  costs.  Value  depreci¬ 
ation  over  time  is  also  handled  as  a  separation  operation  (i.e.,  not  included 
in  the  equations). 


Any  organized  "industrial"  operation  is  a  system  concerned  with  trans¬ 
forming  a  set  of  elements,  called  inputs .  into  some  other  set,  called  a  set 
of  outputs .  in  pursuit  of  its  objective  or  objectives.  An  operation  is  the 
function  carrying  one  set  of  elements  into  another  and  will  be  designated 
as  F(  ).  Our  inputs  are  such  things  as  labor,  information,  material,  etc., 
which  measure  as  dollar  costs  and  our  input  set  of  costs  for  the  i-th  pro¬ 
duct  for  the  period  Of  time  j  is: 


Eq.  18 


f  <eii* 


'121 


*  ^ 

,cin^ 


i 

E 

j»o 


*iJ 


Output,  or  gross  profit,  P,  measured  in  dollars,  is  a  set  of  measures 
produced  by  the  operation  during  some  time  interval,  thus  with  z  as  a 
remainder :  / T  \ 

Eq.  19  F(C.3)  5  -  z±  e  (p^.  P*2 . ^ 

We  will  define  a  term  net  profit  as  the  difference  between  total  grope 
profit  and  total  costs,  so  that  the  worth,  z^,  of  the  i-th  product  which  is: 


Eq.  20  z^ 


T. 

£pM‘P(0iJ) 


becomes  by  this  operation  on  the  costs: 
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Ti 

Eq  •  21  z  j_  =  ^  P  i  j  « 

3=o  J 

Examination  of  all  of  the  costs  indicates  that  some  costs,  namely  the 
ones  associated  with  development  of  the  product  occur  in  an  independent  time 
sequence,  of  length  ti#  from  that  of  the  product  time  sequence,  of  length  T.^. 

VJe  therefore  can  take  the  liberty  of  separating  these  out  without  -change  in 
the  relationship,  thus: 


T.+t. 
1  .i 


T.+t. 
1  1 


Eq.  22  z^  = 


piJ  '  CiJ  '  ^  (pij  '  °i3) 


Another  simplification  is  to  define  p^  and  c^  as  the  average  value 

over  its  time  period  and  multiply  by  the  length  of  its  lifetime,  thus  for 
any  length  of  time  n  >T  +  t 

Eq.  23  -  (Pi.Ti  -  04.^)”.! 

The  objective  of  the  firm  is  to  maximize  all  products  over  all  time, 
both  short-term  and  long-range,  and  this  is  designated  as  follows: 


m 

Eq.  2 U  Maximize  Z  =  T.  (p.  T.  -  c.  t.)1}  , 

i*  1  i*  x  j=l 


subject  to  any  specified  restrictions. 

The  matrix  represented  by  Equation  21+  is  shown  in  Figure  3. 
Fig  3.  Matrix  of  profits  and  development  costs  over  time 


TIME 

j=l - 


Sums  over  j 


i=1  “cll  ~c12  j  ~c] 

I  “Cnn  j  — C, 


It  pl,t+l  “7"  pin  pl*Tl“cl*tl 


321  c22  "  1"  “c2t  jP2,t+l  P2n  P2,T2_C2*t2 


PRODUCTS 


.1  J.  J 


m  j  cml  j  Cm2  "  “cmt 


--  p  p  .T  -c  ,t 
rim  m  m*  m 


Sum  over 


ill 

J  and  i  -  .MPi.VCi.t.)^ 
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For  the  i-th  product,  the  development  costs,  -c.q,  -ci2, 


9 


form  a  basis,  and  ; the  profits,  t  +1’  pi  t  +2* - »  P<  -p 

basis,  but  both  belong  to  the  same  set  of  transformed  inputs 
for  the  i-th  product  vector. 


form  another 
(i.,e. ,  outputs) 


It  is  not  clear  how  we  can  maximize  Z  for  the  firm  until  an  investiga¬ 
tion  is  made  of  the  most  elemental  matrix  that  can  be  formed,  then  of  the 
elemental  matrix  expanded  by  X  in,  the  j-th  direction,  and  then  by  2,  3, 

- ,oo.  We  tan  then  repeat  this  procedure  for  the  i-th  direction,  and 

then  generalize  for  m  products  over  n  years  for  any  firm. 

Suppose  we  consider  the  most  elemental  system  where  we  want  at  least 
one  "-c"  in  every  i-th  row  and  only  one  np"  in  every  j-th  column  after 
the  ,1=1  column.  In  the  entire  matrix,  we  will  simplify  life  by  making  all 
c's  equal  and  all  p's  equal.  By  assuming  the  above,  we  are  saying  that 
there  must  be  a  profit,  p,  in  each  year  after  year  1,  and  that  there  must 
be  a  development,  and  its  cost,  c,  for  every  product. 


The  most  elemental  matrix,  Q22,  is: 


Q. 


'22  ‘ 


-c 


p  -  c 
0-0 

Z  =  p  -  c 


Expanding  Q22  by  1  in  the  j-th  direction,  we  now  find  three  possibilities, 

12  3 

Q23  t  ^23  *  an<*  ^23  * 


*23 


-c 


-c 


p  -  c 

P  ~  C 

2p  -  2c 


-c 


2p  -  c 
0-0 


Z  =  2p  -  c 


Q 


23* 


-c 


-c 


p  -  2c 
P  -  c 

2P  -  3c 


2  1 

As  far  as  the  value  of  Z  is  concerned,  we  can  conclude  that  Q0~ > 
Q2^,  specifically  because  -lc> -2c> -3c. 
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Expanding  by  1  in  the  j-th  direction,  we  find  for  all  possi¬ 
bilities  (totalling  5)  meeting  our  basic  assumptions: 


Z 


p  -  c 
2p  -  c 

3P  -  2c 

2p  -  c 
p  -  c 


3P  ~  2c 


3p  -  c 
0-0 

3p  -  c 


2p  -  2c 
p  -  c 


3p  -  3c 


p  -  3c 
2p  -  c 


?P  -  4c 


We  now  find  that: 

,t-l 


$24  2L  Q 21 ^24^^24* 


324 


If  our  costs  increase 


over  time  by  (l+a)  after  the  first  year  w;e  can  then  also  evaluate  the 
1st  and  2nd  matrices  as  Q^> 


It  should  be  clear  now  that  expanding  by  one  step  at  a  time  until  m 
years,  the  same  results  will  be  obtained  that  the  best  pay-off  is 
v  ..Max 

h*  'V  : 


obtained 


.Max. 

^2n 


-c 


n 


max 


(n-l)  p  -  lc 
0-0 

p(n-l)  -  c(l) 
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The  worst  pay-off  is  obtained  when  a  program  exists  where  zn  =  p(l)  - 
co  L 

c(n-l)(e.g. ,  see  and  Q^),  necessitating  an  additional  program,  to  take 

care  of  supplying  one  and  only  one  "p"  to  the  remaining  years  beyond  the 
first  year,  i.e.  requiring  a  program  where  z2  =  p(n-2)  -c(l).  Thus: 

Zmin  =  *1  *  z2  “  *n  -  c(n  -  1  + 

•  p(n-l)  -  c(n) 

Mxn 

Matrix  represents  the  worst  pay-off 


1  2  n-1  n 


p(l)  -  c(n-l) 
p(n-2)  -  c(l) 


=  p(n-l)  -  c(n) 

=  p(n-l)  -  c(2n-3) 


Let  us  now  extend  the  Qol  matrix  by  1  in  the  i-th  direction. 

*4 

nine  possibilities  are: 


p  -  c 
P  -  c 
c 

Z  -  3P  -  3c 


p  -  c 
2p  -  c 
o-o 


Z  =  3p  -  2c 


The 


2p  -  c 
p  -  c 
0-0 


Z  =  3p  -  2c 


3p  -  C 
0-0 
0-0 


Z  «  3p  -  c 
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I 


)  *r 
I 


* 


1 


p  -  2c 
P  -  c 

P . r. .° 


Z  =  3p  -  4c 


2p  -  2c 
p  -  c 
0-0 


Z  =  3p  -  3c 


Z 


-c 

-c 

— c 

p 

-c 

p 

0 

o 

o 

-c 

p 

0 

Z 


Z 


p  -  2c 
p  -  c 

P  -  c 

3p  -  Uc 


P  -  3c 
p  -  c 
P  -  c 

3p  -  5c 


P  -  3c 
P  -  c 
P  -  c 

3p  -  5c 


,  \t-l 

Considering  future  value  of  costs,  by  the  factor  Cl+a)  „  we  get 
the  following : 


Q3\>Q34>Q34^4>S\>Q31.<lr  <>Q3V  Ql 


Again  we  cane  to  the  same  conclusions  as  before,  i.e.,  that  the  program 
represented  by  Q^.  is  the  best ““Maximizes  total  profit  over  time  to  the  firm 

>4  g  n 

and  the  first  line  of  or  will  give  us  the  worst  profit  picture. 
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Extending  the  picture  step-wise  in  the  i-th  direction  until  m  projects 
are  reached,  we  still  have  the  matrix  as  the  best,  but  repeating  to 

show  m  projects  over  n  years: 


i 

2 

- , 

n 

1 

-c 

P 

p 

p(n-l)  -  c(l) 

2 

0 

o 

d 

0-0 

1 

1 

i 

i 

i 

1 

1 

1 

1 

1 

1 

m 

o 

o 

— 

0 

0-0 

Zmax  *  »(n-l)  -  c(l) 
regardless  of  the 
magnitude  of  m(m  =  l) 


The  worst  picture  is  shown  by 


p(l)  -  c(n-l) 


p(n-2)  -  c(m-l) 


p(l) +p(n-2)-c(n-l) 
-c(m-l) 

p(n-l)-c(n+m-2) 
where  m  <  n-1 


when  m=n-l,  Z  .  =  p(n-l)-c(2n-3) 


The  preceding  illustrations  leading  to  the  development  of  maximum 

pay-off  of  Z  =  p(n-l)  -  c(l)  should  be  recognized  as  manipulation  of 
max 

the  number  of  p's  and  c's  as  variables  so  that  all  possibilities  could  be 
examined  and  understood.  However,  given  a  proposed  program,  with  fixed 
estimates  of  product  life  and  development  time,  how  shall  we  calculate 
little  z^  for  the  project  so  that  the  sum  of  all  z^'s  will  give  us  the 

Z  for  the  whole  program?  We  have  already  derived  the  strategy  as 
max 

shown  by  equations  7  and  8  and  figure  1. 
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In  order  to  be  rigorous,  we  will  use  our  profit  and  cost  matrices,  and 
the  results  derived  from  them,  for  developing  the  best  product  strategy. 

We  must  first  designate  actual  times  for  the  i-th  project  as  T.  for  the 
number  of  years  of  profit  taking  where  T.<  n-1  and  as  t.  for  tfee  number  of 

years  during  which  the  development  costs  are  incurred  where  t.  also  is 

<  n-1.  T.  +  t.<  n.  1 

-  11= 

It  appears  inevitable  that  we  must  recognize  as  a  reference  point  the 
inherent  measure  of  central  tendency  of  time  within  the  set  of  all  similar 
products.  This  was  discussed  with  reference  to  the  physical  phenomena  of 
Alfrey's  internal  timescale  (relaxation  time),  r,  where  the  experimental 
timescale  was  ©  and  the  original  property  of  the  material  varied  by  the 

factor  e~^r.  We  ^similarly  recognize  an  internal  time  £cale  as  the  aver¬ 
age  product  life,  T,  and  the  average  development  time,  t,  for  a  class  of 
related  products ._  Historical  data,  or  at  least  comparisons,  supply  the 
values  for  T  and  t. 

Let  us  use  the  elemental  matrices  to  derive  the  optimal  strategy,  and 
the  arbitrarily  assigned  values  of  T  =  1  Mid  t  =  1.  For  a  matrix  Q19  or 
Q92  (see  above),  we  get  2  =  z.  =  p  T  -  c  t  =  p  T  -  c  t.  In  (also  see 

above J ,  T  =  2  T  and  results  in  the  saving  of  another  development  project 
and  its  cost,  c,  over  the  situation  where  projects  of  average  product  life 
and  development  time  would  have  existed,  as  in  Q*.  In  q|.  ,  we  again  ex- 

perience  the  saving  of  one  development  cost  and  in  Q^, ,  we  save  two  devel- 

mm  -^4  """ 

opment  costs  when  T  =  3  T.  Thus  by  further  extension  to  the  Q™  matrix, 
we  have  proven  that  when  T  =  n  T  we  save  (n-l)  c  cdsts.  n  =  T/T  so  that 
the  saving  is  (T/T-l)-c  over  the  situation  where  only  average  projects 
exist.  Further,  let  us  not  forget  that  the  number  1,  for  one  c,  must  be 
inserted,  or  instead  we  should  substitute  its  true  symbol,  t,  thus  we  get 
that  the  saving  is  c  (l)  (t/t  -  1)  or  c  t  (T/T  -  l).  By  way  of  further 
explanation,  we  will  now  say  that  the  normal  or  average  project  has  t  =  2 

Q§  and  the  actual  case,  q7,  as 
*-k  24 


p  -  2c 
p  -  2c 


=  2p  -  4c 


2p  -  2c 
o-o 


Z  =  2p  -  2c 

We  will  note  that  c  t  (T/T  -  l)  =  c(2)  (2  -  1)  =  2c  which  is  borne  out 
by  comparison  of  Q|^  and 


and  T  *  1. 
follows : 


The  normal  matrix  is  thei 


*24* 


-c 


-c 


-c 


-c 


«L,= 


-c 


-c 
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Proof  of  the  extension  to  the  n-th  case  for  any  T  and  T  is  obvious. 
Further  proof  that  the  increase  of  the  value  of  p  due  to  decrease  of 
development  time  below  the  average  time  t  is  pT  (1  -  t/t)  would  merely 
be  an  exercise. 


Thus,  where  the  average  project  value  is: 


Eq.  25 

zi  =  pi*T  “  ci*t  =  pi*^  "  ci*^ 

the  extended 

formula  for  any  T  or  t ,  given  any  T  and  t  is : 

Eq.  26 

z.  -  P-.T  +  p..?(l  -  t/t)  -ci#t  +  ci#t(T/T  -  1) 

Eq.  27 

z.  =  p..  [t  ♦  T  (1  -  t/E)]  -  c.#[tH  (1  -  T/fj] 

Compare  with  equations  7  and  8. 

If  T.*  = 
i 

[t  +  T(1  -  t/t2 

and  t.*  = 

l 

[t  +  t(l  -  t/t)] 

then  we  have  proven  that  to  maximize  Z,  the  pay-off  or  profit  -to  a  firm, 
we  must  use: 


n 

j=l 


By  proof  that  we  obtain  the  maximum  pay-off  for  n-x,  n-x+lj - ,  n 

times  and  for  m-y,  m-y+1, - ,  m  products,  as  well  as  if  F  «  t  ■  1,  2, 

- W|  and  for  any  strategy,  i.e.,  one  and  one  p  in  each  column  of  the 

matrix,  only  two  p,  only  three  p,  etc.,  we  have  proven  that  Eq  11  applies 

to  any  selection  or  evaluation  of  projects,  prior  to  starting  or  during 

development.  Discounting  of  dollars  for  their  present  value  (for  profits) 

or  future  value  (for  costs)  does  not  deny  the  proof,  but  supports  it  and 

makes  Z  more  meaningful.  Lastly,  it  should  be  obvious  that  c  surid 
max  i  j 

p_^j  can  now  be  any  possible  value,  i.e.,  not  necessarily  equal  as  in  the 
elemental  matrices. 


To  make  the  model  more  general  and  exact,  we  introduce  x^,  where 
x.  =  o  or  1  depending  on  whether  we  do  not  select  or  do  select  a  project  i, 
respectively.  We  thus  have  Eq  29* 


max 


c. 

l 


t.*) 

l 


n 

j=l 


Eq.  29  Z 
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As  in  Figure  A  (Ref  7),  for  a  group  of  projects,  x.  =  o  or  1,  and 

subject  to  any  specific  restrictions,  we  select  G..  by  inspection  out  of 
the  possible  groups  or  combinations  of  all  projects . 


Group  Z  Total  Costs 


Fig  4°  Selection  of  group  of  projects. 


For  a  large  number  of  projects,  the  number  of  combinations  would  be 
extremely  difficult  to  handle,  therefore  it  is  simpler  to  list  the  indi¬ 
vidual  projects  in  a  decreasing  order  of  pay-off  as  determined  by  z 
(Equation  27)  and  successively  select  the  project  with  the  highest  z  value, 
next  highest,  etc,,  testing  each  time  for  compliance  of  the  group  thus 
being  formed  step-wise  against  any  restrictions. 

To  illustrate  the  application  of  our  proposed  method,  let  us  generate 
some  projects  randomly  as  shown  in  Table  1,  The  average  years,  T  and  t, 
are  equal  to  3,  We  will  also  place  the  restriction  on  our  problem  that  not 
more  than  three  development  projects  can  be  handled  in  any  one  year,  because 
of  our  fixed  staff  of  three  teams  of  engineering  specialists.  There  are  no 
restrictions  on  budget  and  our  formula  will  be  used  as  a  measure  of  profit 
for  one  example,  whereas  the  second  example  for  comparison  will  use  straight 
computation  of  profit.  There  is  no  restriction  on  production  of  number  of 
products.  Table  2  shows  the  computations  of  Z  and  straight  profits  for 
five  projects  appearing  in  each  of  five  successive  years.  Projects  not 
started  in  any  year  become  useless  in  future  years,  but  projects  once  started 
must  be  re-evaluated  in  each  successive  year — competing  with  new  proposed 
projects.  An  old  project  is  likely  to  be  canceled  by  a  new  one  if  its  re¬ 
maining  value  is  less  than  that  of  a  new  project,  after  a  10  %>  margin  of 
error  in  estimating  is  allowed.  The  figures  have  already  been  discounted. 

Tables  3  and  4  show  the  computations  and  that  our  recommended  method 
is  better  by  12,250  -  11,  450  =  800  thousand  dollars,  or /80,000/year 
over  10  years. 


TABLE  1 


Generation  of  Hypothetical  Projects,  T  -  t  -  3 


* 

Project  Factors  | 

Cum 

.  F  of 

Random 

T, 

C| 

»l 

Occurrence 

Number 

In  $1000 

In  yrse 

In  $1000 

In  yrs . 

of  p,t,c,t, 

Groups 

350 

i 

90 

i 

0.2 

1-2 

400 

2 

120 

2 

0.4 

3-4 

450 

3 

150 

3 

0.6 

5-6 

500 

4 

180 

4 

0.8 

7-8 

550 

5 

210 

5 

1.0 

9-0 

Random 

Numbers1 

■l 

IB 

V 

Project  No. 

6  8  4  7 

450 

4 

120 

r 

4 

3 

3 

i 

26  9  4 

IBS 

3 

210 

2 

4 

2 

2 

8  5  15 

500 

3 

90 

3 

3 

3 

3 

1110 

350 

1 

90 

5 

-1 

B 

No  Profit 

16  5  0 

350 

3 

150 

5 

1 

If 

No  Profit 

26  4  5 

350 

3 

120 

3 

3 

Bp 

4 

9  5  2  7 

550 

3 

90 

4 

2 

Kfl 

5 

6  7  8  9 

mm 

4 

180 

J 

2 

6 

97  3  4 

550 

4 

120 

5 

i 

7 

7  3  2  0 

500 

2 

90 

■1 

No  Profit 

7  5  2  4 

500 

3 

90 

■9 

8 

6  4  0  5 

450 

2 

210 

• 2 

9 

2  6  8  9 

350 

3 

180 

i 

No  Profit 

4542 

400 

3 

120 

1 

5 

l 

10 

0139 

550 

1 

120 

5 

No  Profit 

87  37 

500 

4 

120 

4 

11 

20  1  1 

350 

5 

90 

1 

-1 

12 

617  5 

550 

1 

180 

3 

1 

5 

No  Profit 

1947 

350 

5 

120 

4 

2 

13 

36  16 

400 

3 

90 

3 

3 

14 

4  5  2  4 

400 

3 

90 

2 

2 

15 

4  194 

400 

1 

210 

n 

No  Profit 

96  38 

550 

3 

120 

Bfl 

16 

3467 

400 

2 

150 

1 

5 

No  Profit 

24  80 

350 

2 

180 

5 

0 

6 

No  Profit 

2  3  2  0 

350 

2 

90 

5 

0 

6 

No  Profit 

38  3  1 

400 

4 

120 

1 

0 

17 

640  3 

450 

2 

210 

2 

3 

18 

3669 

400 

3 

150 

5 

1 

5 

No  Profit 

3  5  3  0 

400 

3 

120 

5 

1 

5 

No  Profit 

6866 

450 

4 

150 

3 

2 

19 

-9  0  5  5 

550 

5 

150 

3 

1 

20 

129 
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TABLE  2 

Computations  of  Z  and  Straight  Profit  (In  $1,000  units) 


Year 

Project 

PlV 

Vl* 

z. 

P,Tl 

Vl 

1 

1,350 

360 

990 

<* 

1,800 

480 

2 

1,440 

420 

1,020 

1,050 

420 

1 

3 

1,500 

270 

1,230  , 

1,500 

270 

4 

1,050 

360 

690 

1,050 

360 

5 

1,100 

360 

740 

1,650 

360 

6 

900 

720 

180 

1,800 

900 

7 

2,750 

120 

2,630 

2,200 

240 

2 

8 

2,000 

180 

1,820 

1,500 

180 

9 

900 

840 

60 

900 

630 

10 

2,000 

120 

1,880 

1,200 

120 

11 

1,500 

360 

1,140 

2,000 

480 

12 

2,450 

-90 

2,540 

1,750 

90 

3 

13 

1,400 

240 

1,160 

1,750 

480 

14 

1,200 

270 

930 

1,200 

270 

15 

1,600 

180 

1,420 

1,200 

180 

16 

1,100 

480 

620 

1,650 

480 

17 

2,400 

0 

2,400 

1,600 

120 

4 

18 

1,350 

630 

720 

900 

420 

19 

1,800 

300 

1,500 

1,800 

450 

20 

2,750 

150 

2,600 

2,750 

450 

21 

3,150 

-180 

3,330 

2,250 

180 

22 

1,050 

540 

510 

700 

360 

5 

23 

1,200 

540 

660 

1,200 

540 

24 

1,650 

540 

1,110 

550 

180 

25 

2,250 

210 

2,040 

1,800 

420 

Total 

1,320 

630 

1,230 

690 

1,290 

900 

1,960 

1,320 

270 

1,080 

1,520 

1,660 

1,270 

930 

1,020 

1,170 

1,480 

480 

1,350 

2,300 

2,070 

340 

660 

370 

1,380 
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TABLE  3 

Example  Using  Recommended  Method 


Year 

P  reject 

p.v 

ciV 

X 

Notes 

1 

3 

1,500 

270 

& 

1,230 

2 

1,440 

420 

1,020 

1 

1,350 

360 

.990 

3 

2,000 

180 

1,820 

2 

1,750 

210 

1,540 

Canceled  by  7 

2 

1 

1,800 

240 

1,560 

Canceled  by  10 

7 

2,750 

120 

2,630 

10 

2,000 

120 

1,880 

Finished  in  2nd 

Year 

3 

2,500 

90 

2,410 

Finished  in  3rd 

Year 

3 

7 

3,300 

0 

3,300 

Finished  in  3rd 

Year 

12 

2,450 

-90 

2,540 

Finished  in  3rd 

Year 

20 

2,750 

150 

2,600 

4 

17 

2,400 

0 

2,400 

Finished  in  4th 

Year 

19 

1,800 

300 

1,500 

21 

3,150 

-180 

3,330 

5 

20 

3,300 

300 

3,000 

19 

2,250 

150 

2,100 

A.  Computations  and  Decisions 

1 

Year 

Project 

4 

1  2  3 

4 

5  6 

7  8 

9  10 

3  -90  -90  -90  +500  +500  +500 

2  -210  -210  +350  +-350  +350  Cancel 


1 

-120 

-120  -120 

-120 

+450 

+450 

+450 

+450 

Cancel 

7 

-120  -120 

+•550 

+550 

+550 

+550 

10 

-120  +400 

+400 

+400 

12 

-90 

+350 

+350 

+350 

+350 

+350 

20 

-150 

-150 

-150 

+550 

+550 

+550 

-too 

17 

-120 

400 

400 

400 

400 

19 

-150 

-150 

-150 

+450 

+450 

+450 

+450 

21 

-180 

+450 

+450 

+450 

+450 

+450 

Profits 

400 

1,800 

2,200 

2,250 

2,750 

2,200 

1,450 

1,450 

14,500 

Costs 

-420 

-330  -300 

-420 

-480 

-300 

2,250 

12,250 

B. 

»  Charting  by 

Year 

TABLE  4 

Example  Using  Straight  Computations 
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Year 

Pro|ect 

piTi 

Vi 

Total 

Notes 

1 

1,800 

480 

1,320 

- 

1 

5 

1,650 

360 

1,290 

3 

1,500 

270 

1,230 

1 

1,800 

360 

1,440 

5 

1,650 

270 

1,380 

2 

3 

1,500 

180 

1,320 

Cancelled  by  7 

7 

2,200 

240 

1,960 

1 

1,800 

240 

1,560 

X 

5 

1,650 

180 

1,470 

Cancelled  by  12 

J 

7 

2,200 

120  • 

2,080 

Finished  in  3rd  Year 

12 

1,750 

90 

1,660 

Finished  in  3rd  Year 

1 

1,800 

120 

1,680 

Finished  in  4th  Year 

4 

20 

2,750 

450 

2,300 

17 

1,600 

120 

1,480 

Finished  in  4th  Year 

20 

2,750 

300 

2,450 

5 

21 

2,250 

180 

2,070 

25 

1,800 

420 

1,380 

A*  Computations  and  Decisions 


Yaor 


Pro|ee» 

1  2 

3 

4 

s 

6 

7 

8 

9 

10 

1 

-120  -120 

-120 

-120 

+450 

+450 

+450 

+450 

5 

-90  -90 

-90 

“90 

+550 

+550 

+550 

Cancel 

3 

-90  -90 

“90 

+500 

+500 

+500 

Cancel 

7 

-120 

-120 

+550 

+550 

+550 

+550 

12 

-90 

+350 

+350 

+350 

+350 

+350 

20 

-150 

-150 

“150 

+550 

+550 

+550 

+550 

17 

-120 

+400 

+400 

+400 

+400 

21 

-180 

+450 

+450 

+450 

+450 

+450 

25 

-210 

-210 

+450 

+450 

+450 

+450 

Profits 

900 

1,750 

2,300 

3,200 

2,650 

1,450 

1,450 

13,700 

Costs 

-300  -330 

“330 

“390 

-540 

“360 

2,250 

11,450  Total 

B.  Charting  by  Year 
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ABSTRACT 


The  following  general  problem  is  considered:  A 
random  variable  x,  called  the  attribute  variable,  distributed 
with  a  known  frequency  function  and  possibly  contaminated 
with  additive  independent  random  error  to,  which  is  also 
distributed  with  known  frequency  function,  is  subjected  to 
a  test  that  determines  whether  or  not  (x  +  w)  lies  within  a 
specified  interval;  the  interval  may  have  end  points  which 
are  themselves  independent  random  variables  with  known 
frequency  functions.  What  is  the  conditional  frequency 
function  of  x,  given  that  (x  +  w)  has  passed  the  test? 

An  expression  for  the  solution  of  the  general  prob¬ 
lem  is  given.  The  problem  is  particularized  to  various 
special  problems  by  imposing  restrictions  on  the  attribute 
random  variable,  the  error  random  variable,  etc.  A  systematic 
method  is  developed  for  imposing  these  restrictions  which 
involves  the  use  of  Dirac  delta  functions.  Solutions  to  the 
specialized  problems  are  obtained  utilizing  specific  fre¬ 
quency  functions,  and  a  numerical  example  is  presented. 


I.  INTRODUCTION 

Here  are  many  engineering  system  test  procedures  which  require  that  some  attribute  of  the  system  under 
test  be  within  certain  tolerance  limits.  Go/no  go  tests  of  the  output  voltage  of  electronic  systems  fall  into  this 
category,  as  do  many  quality  assurance  tests  used  in  industry.  Since  there  are  several  sources  of  uncertainty  in 
such  test  procedures  (uncertainty  about  the  exact  location  of  the  tolerance  limits,  uncertainty  associated  with  the 
attribute  being  tested,  etc.),  it  is  important  to  make  some  estimate  concerning  just  what  information  can  be 
gleaned  from  these  tests. 


Pug*  1 
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The  problem  of  forming  estimates  may  be  stated  probabilistically  in  a  very  general  way.  The  purpose  of 
this  paper  is  to  formulate  and  solve  this  general  problem  and  to  apply  the  solution  to  a  number  of  special  cases. 
It  should  be  emphasized  that  the  approach  employed  is  based  on  sample  statistics  and  not  on  time  series  analysi; 
and  the  assumption  which  holds  throughout  the  analysis  is  that  the  random  variables  discussed  are  time  invariant 


II.  DISCUSSION 


The  discussion  which  follows  makes  use  of  some  mathematical  tools  which  may  have  become  unfamiliar 
to  the  reader  through  disuse.  For  this  reason  a  short  refresher  on  the  properties  of  frequency  functions  is  include 
as  Appendix  A.  In  explicit  terms,  the  question  considered  here  is  the  following:  a  random  variable  x,  called  the 
attribute  variable,  distributed  with  a  known  fr .  f.  and  possibly  contaminated  with  additive  independent  random 
error  w,  also  distributed  with  known  fr.  /.,  is  subjected  to  a  test  which  determines  whether  or  not  (*  +  w)  lies 
within  a  specified  interval;  the  interval  may  have  end  points  which  are  themselves  independent1  random  variables 
with  known  ft. \  f.* s.  What  is  the  conditional  fr.  f.  of  x,  given  that  (*  +  tv)  has  passed  the  test;  i.e.,  given  that 
(*  +  w)  lies  in  the  distributed  test  interval?  For  example,  one  preflight  check  of  a  guidance  system  might  be  the 
examination  of  a  nominally  zero  (constant)  guidance  voltage  with  a  circuit  which  gives  an  OK  to  fire  indication  if 
its  input  lies  in  a  preset  interval.  It  is  assumed  (1)  that  the  voltage  of  interest  has  a  known  fr.  f.,  (2)  that  an  etroi 
voltage  (with  known  fr.  f.)  appears  somewhere  between  the  guidance  system  and  test  circuit  and  adds  to  the  volta< 
of  interest,  and  (3)  that  the  limits  of  the  test  interval  are  distributed  with  known  fr.  f.' s;  that  is,  on  a  given  test 
the  tolerance  limits  are,  in  general,  different  from  the  nominal  limits  due  to  inaccuracies  in  the  measuring  device. 
Thus  an  OK  to  fire  indication  gives  the  information  that  the  error-contaminated  attribute  variable  lies  between 
these  two  essentially  unknown  limits.  This  situation  is  shown  schematically  in  the  following  sketch  (it  should  be 
recalled  that  the  voltages  are  assumed  constant  for  the  period  under  study). 


OK  if 

U  <  *  +  w  <  V 


l 


See  footnote  page  3. 
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We  wish  to  determine  what  information  an  OK  signal  provides  about  the  fr.  f.  of  x\  equivalently,  we  wish 
to  compute  the  probability  p(*i  <*<*2  |  «*<*+«><  v)  that  *  lies  in  the  specified  interval  (*|,  *2),  given 
that  (x  +  u>)  has  passed  the  test.  Defining  *  +  w  =  z  and  denoting  events  A  and  B  as  follows: 

Ax  xY  <  *  <  *2 
B :  uj  <  z  <  v 

the  desired  quantity  p(*j  <  *  <  *2  I  «*  <  *  •<  v)  becomes 

p(A  |  fl)  -  pU^fl)  (1) 

p(B) 


the  equality  being  exactly  the  definition  of  conditioned  probability  (Ref.  1,  Sec.  2.8,  page  29).  The  fr.  f.9s  of 
w9  ut  and  v  are  assumed  known: 


*  -  fx{£) 

w  ~  sj0*) 
u  ~  hu(rj) 
v  ~  Av(v) 


(2) 


Because  these  variables  are  all  mutually  independent,2  their  joint  fr.  f.  is  the  product  of  their  separate  fr .  f.fe: 


x,  w9  u,  v  ~  fx(€)guS*>)  hu(r})hviv)  =  p(x  =  ^  tv  -  co^u  =  rj^v  =  v) 


(3) 


What  is  needed  to  determine  the  quantities  in  Eq.  (1)  is  not  the  joint  fr.  f.  of  x9  w9  u9  and  v  but  rather  that  of  x9  z9 
u9  and  v . 

It  is  a  well-known  theorem  of  statistics  (Ref.  2,  Sec.  22.2,  page  292)  that  if  x  and  w  have  joint  fr.  f. 


fJL€)gwi») 


the  random  variables  x  and  z  =  x  4-  w  have  joint  fr.  f.  (evaluated  at  x  ~  £9  z  «  £). 


2If  u  and  v  are  not  independent  but  have  a  known  joint  fr.  f.  hu v(ti9  v),  the  factor  hjirf)  hj,v)  is  simply  replaced 
throughout  this  report  by  hu  v(i)9  v);  the  separate  integrals  with  respect  to  i)  and  v  in  Eq.  (7)  et  seq  then  combine  into  one 
double  integral. 


Page  3 
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It  is  clear  that,  as  might  be  expected,  x  and  z  are  not  independent  even  though  x  and  w  were;  and  the  presence  of 
the  factor  hu(rj)hv(v)  does  not  affect  the  argument.  Thus,  the  joint  fr .  /.  of  xf  z,  u,  and  t;  is  given  by 


x,  z  «  x  +  w9  u,  v  ~  fx(€)gw(l  -  £)hu(r})hv(v) 


(4) 


Integrating  over  all  £  provides  the  joint  fr.  f.  of  zf  u9  and  v  (Ref.  1,  Sec.  4.5,  page  82) 


**  «*,  v 


“  &i*\  hu<V)hv(v) 


(5) 


By  direct  application  of  the  definition  of  joint  fr. 


■  p(xj  <  X  <  *2^"  •»  .<  Z  <  oo^  -  oe  «<  U  <  «^|  <  V  <  op) 

/°  C  “«*  fvm*  C ym 00 

-J  4<a  J  8w(t-&  /  ab(t?)  /  hjiv)  i  V  d  7)J  Li  g 

*4  **|  •'{■-00  't)  O-J,  •'|/“* 


(6) 


(7) 


Similarly, 


p(B)  ■  p(—  oo  .<  *  <  ooO—  eo  <  u  .<  i^i  <  »  <  «) 


(8) 


/*(£)*„(£  -  a 


A„(v)  d  v  d  t)d  L 


(9) 


Equations  (7),  (9)  (with  the  order  of  integration  altered),  and  (1)  give 
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(144) 


»*2  /°  C /°VmL  /°V»og 

/  /*(£)/  gJt-€)  ku(r))  hv(vUvdr)didg 

J  g  15  JC 1  o'  £  *  -00  7j  *  — ©o  o'  v  *  £ 


pU  |  B)  =  p(*j  <  *  <  *2  lu  ^  *  <  w)  = 


/>£«<»  />£«oo  />  TJ  *  £  /»V*oo 

/  fxW  gJl-£)  4b(t,)/  hv(v)dvdrjdid^ 

o/gm-oa  J  £■-<»  0/77  «  “00  «/V"£ 


(10) 

Use  of  the  extension  of  Eq.  (A2)  (Appendix  A)  to  conditional  /r.  /.’a  gives  the  /r.  /.  of  *  conditioned  upon 
event  B  (B  is  i»  .<*  =  *  +  «'<  v),  denoted  fx  |fl(£): 


4  |»(f)  -  /«<£> 


/£  *»  /®T)  «  £  />  r  “00 

«u,(C  -  &  I  hu(-q)  hv(v)  dvdrjdt, 

£*=-00  <Jt\  c  -00  J  v  *  £ 

/*r"ofl  /°£“<»  /» 77  *  £  /°  v “00 

/  f» W/  «u,(C  -  T>l  ku(y)  hv(v)dvdr)didr 

O'  T  *  -OO  ®'£*-oo  o' 17  *  -00  0/  V  *  £ 


(10a) 


Note  that  £  is  not  used  as  a  variable  of  integration  in  Eq.  (10a)  (vis  substituted  in  the  denominator). 

Equation  (10)  represents  the  solution  of  the  general  estimation  problem  posed.  The  problem  may  be 
particularized  to  various  specific  problems  by  imposing  restrictions  on  the  random  variables  u,  v,  and  z  and  their 
frequency  functions.  For  example,  if  u  and  v  are  allowed  to  be  random  variables,  and  if  z  =  x,  i.e.,  w  =  0,  the 
problem  which  is  discussed  in  Ref.  3  is  described.  If  u  and  v  are  required  to  be  constants  uQ  and  Vq  such  that 
w0  -  *1  aiM^  u0  ^  x2 *  Qn(f  if  *  -  x>  the  classical  confidence-limit  problem  is  described  (Ref.  1),  A  systematic 
approach  is  available  for  imposing  these  restrictions  which  involves  the  use  of  Dirac  delta  functions.  It  is 
possible  to  think  of  a  constant  as  being  a  “distributed”  variable  whose  fr .  f.  is  a  delta  function  centered  at  the 
nominal  value  of  the  constant.  Thus,  replacement  of  the  appropriate  fr.  f.' s  in  Eq.  (10)  by  delta  functions  leads 
in  a  very  simple  way  to  solution  of  the  associated  special  problem.  It  is  easily  demonstrated  that  the  5  function 
is  indeed  the  frequency  function  of  a  constant.  (The  validity  of  the  approach  may  be  verified  by  reference  to  the 
alternate  derivations  of  the  equations  which  follow  in  Appendix  B.)  Thus,  if  a  is  a  constant,  it  may  also  be 
viewed  as  the  random  variable  y,  where 


y  ~  S(y  -  a). 
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Then  the  probability  that  y(i.e.  a)  lies  in  any  interval  is  clearly  1  if  a  lies  in  the  interval,  and  0  otherwise.  But 


f*2  5u  \j  _  I1  if  yi  ^  a  ^ 

lyi  y  ~  °  y  lO  otherwise 


X2 


■  p(y i  <  y  <  y<j) 


It  is  also  clear  that 


a)  dy  =  1 


The  general  estimation  problem  involves  additive  error  and  distributed  limits.  This  case  reduces  to  that 
of  distributed  limits  alone  by  the  substitution  in  Eq.  (2)  et  seq  of 


w  BuS03)  =  Mo)  -  0)  =  S(oj) 


(11 


This  substitution  is  equivalent  to  die  statement  that  the  additive  error  w  is  identically  zero;  z  then  reduces  to 
x,  and  Eqs.  (10)  and  (11)  combine  to  yield 


p(*l  <  *  <  *2  1“  —  *  —  = 


/°f  ■*  2  /°c«~  /°*>*C  /°  V  "oo 

J  fx(&l  Hi-O  •  hu(rj)  I  hv(v)  dvd-qdU€ 

•  €  m*l  O'  {*-00  o' 7)  “  -OO  «'v*{ 


(Same  as  numerator  with  *j  =  -  o»,  =  °°) 

»f-*2  /’’I'f 


“*2  /°V  'f  /» v»<» 

/*(£)  /  hu(-q)  /  hv(v)  dvdrjdg 

i  m%i  0/17*00  ©/v*£ 

(Same  as  numerator  with  “  °°) 


(IS 


(ia 
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The  fr.  f.  of  x  conditioned  on  event  B  (  here  B  becomes  u  <  x  <  v)  denoted  fx  | %(£;)  is  given  by 


f>  V  m(  f>V<*ao 

/  M7?)  / 

o/  f)  "-00  Jvm£ 


(v)  dvdr\ 


/oo  fj)  ■  T  y°  v*t» 

/*(^  /  *„(*> )/  4v(v)  dvdr\dr 

oo  •'tl  *“oo  *  VmT 


(13a) 


The  note  following  Eq.  (10a)  applies. 

An  alternate  derivation  of  Eq.  (13),  not  dependent  on  the  sneaky  behavior  of  8  functions,  is  given  in 
Appendix  C. 

The  following  substitutions  give  rise  to  the  case  wherein  the  test  limits  are  known  with  certainty,  but  the 
attribute  random  variable  x  is  contaminated  with  additive  error  w: 


u  ~  hu(-q)  =  8(77  -  uq) 


v  ~  ^(v)  .  8(u  -  v0) 


(14) 


Equation  (10)  becomes 


pix i  <  *  <  *2  l“o  V 


/°{mx  2  /°  i  /“r!  /»v»~ 

/  /  eji-  ft  8  (*?  -  «*o)  /  8  (v  -  «0)  dvdTjdtdg 

J  €  "*!  J  r\  m  “t»  c/v®  c 

(Same  as  numerator  with  «  —  oof  *2  =  00 ) 


(15) 


Consider  the  integration  with  respect  to  Tj  in  the  numerator  of  Eq.  (15).  It  is  independent  of  the  remainder  of  this 
expression,  and  its  value  is  zero  unless  the  argument  of  the  8  function  reaches  zero  —  that  is,  unless 
-  *°  <  «0  Similarly,  the  integration  with  respect  to  v  yields  zero  unless  £  <  oq  <  00.  Thus  there  is  no 

contribution  to  the  numerator  unless  uQ  <  £  <  vq,  and  the  limits  of  the  integration  with  respect  to  £  may  be 
confined  to  the  interval  (uq,  Vq).  This  argument  holds  for  the  denominator  as  well.  Equation  (16)  follows 
immediately. 


Page  7 


Jet  Propulsion  Laboratory 


Soctian  Report  No*  12-16* 


p(*i  <  x  <  x2  luo  ^  z  <  wo)  ~ 


r  ”  w 


£  = 


"0 


8jl-&dU€ 


'“0 


I  fj&l 

=  «/c=Ua 


ffjs  -  <f> 


(1/, 


1/ 


(16 


The  /r.  /!  of  *  conditioned  on  event  B  (here  fl  is  u0  <  z  =  x  +  w  <  vQ),  denoted  fx  |fl(£),  becomes 


I 


W  -  >f)  rfS 


/°r  3“  /"^"o 

/  4(tM  gw(i-r)dCdT 

O'  T  «  -oo  ^  J«ur 


(i6u; 


The  note  following  Eq.  (10a)  applies. 

Equation  (10)  reduces  to  the  simplest  case,  that  of  the  classical  confidence  limit  problem,  upon  the 
substitutions: 


w  ~  «„,(")  =  8(o>) 

«  ~  ^^(77)  =  h(rj  -  u0)  ► 

v  ~  Av(i/)  =  S(i/  -  v0) 

and  the  specification  that  *j  and  *2  be  within  («0,  t»0).  Equation  (10)  becomes: 


(17) 


p(*i  <  *  <  *2  l«o  <  *  <  f0)  = 


"*2  /°£“«  /°Vmi  y®  V  «<x> 

f*w  8(C-f)/  8(77-  U0)/  S(v  -  v0)dvd-T)dU€ 

f  c*|  »  t B  “»  o/t;  «-oo  ©/  v  *  £ 


(Same  as  numerator  with 


»,  *2  «=  «) 


(18) 
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(i m) 


The  reduction  of  the  numerator  is  as  follows: 


5(£ 


/%-C  /%-. 

-a/  5(7,  -y/ 

®TJ  *  -no  "  V  m  c 


Hv-VQ)dvdT)dU€ 


s  (i-Odidg 


(19) 


as  in  Eq.  (16),  but  the  integral 


s(£  -  a  dc  =  i 


(20) 


since  Uq  <  £  <  Wq. 

The  numerator  becomes 


fJL€) 


(21) 


The  denominator  is  reduced  similarly. 


/°V  *  C  PV~m  /°f "»  /®C*‘t^0 

5(£-a/  8(7,- O0)/  Hv-v0)dvivdld£=  fx(£)  I  Hi-£)didg 

Jl)  ■  -00  *'v*C  Jg  *- eo  </  { «■ 


(22) 


The  integral  on  the  right  side  of  Eq.  (22)  has  values  only  in  the  interval  (u0,  t>0)  by  reason  of  the  definition  of  the 
delta  function.  It  may  be  replaced  by  the  equivalent  relation 
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(1/. 


(23 


Equation  (18)  becomes 


p(* i  <  *  <  *2  l“0  *  £ 


fx(&  d£ 


(24 


TTie  /r.  /■.  of  *  conditioned  on  event  B  (where  B  is  now  Uq  <  *  <  oq),  denoted  /„  |fl(£).  is 


(24a) 


The  note  following  Eq.  (10a)  again  applies. 

It  is  perhaps  instructive  to  make  the  preceding  discussion  more  concrete  by  assigning  specific  functional 
forms  to  the  fr.  f.'e  and  applying  them  to  the  example  outlined  on  page  2  of  this  report.  Suppose  the  nominally 
zero  guidance  voltage  has  a  rectangular  distribution  about  zero  with  end  points  at  ±1  volt.  Thus 


—  -  1  <  i  <  1 
2 


*  ~  /*(£)  =  < 


0  otherwise 

Let  the  error  voltage  be  normally  distributed  about  zero  with  standard  deviation  crw. 


(25) 
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~  s o.  O  = 


>/  2ljcr', 


exp 


2  o2 
w 


(26) 


Let  the  limits  of  the  test  interval  be  normally  distributed  about  the  nominal  limits  Ug  and  Vq  with  standard 
deviations  &u  and  &v. 


“  ~c£(l?;  1*0,  crj  =  hu(rj) 
V  ~0(v;  V0,  crv)  =  hv(v) 


(27) 


Equations  (26)  and  (27)  combine  with  Eq.  (10)  to  give 


p(* l  <  *  <  x2\u  <  z  <  v)  = 


*=^2  /°v  “  £  /°V=oa 

fx(€) J  0 ( C  —  0.  <\)  /  0(^7;  u0cr|l) /  0 (*7  VQ.crJdvdTidtdg 

g  B*t  °kc”oo  *  —  oo  V  °  £ 


(Same  as  numerator  with  =  -  oo,  *=  ®°) 


(28) 


It  is  required  that  +  *  (the  problem  is  soluble  without  this  restriction,  but  attention  here  will 

be  confined  to  this  case);  then  fx(£)  is  simply  Vi  in  the  expression  in  the  numerator;  the  integrand  in  the 
denominator  is  zero  unless  —  1  <  £  <  +  1,  so  that  the  interval  of  integration  with  respect  to  £  may  be  taken  to 
be  (—  1*  !)•  Over  this  interval  fx({ f)  is  %9  cancelling  the  Vi  in  the  numerator  and  giving 


p(x  j  <  x  <  %2  |«  <  z  <  v)  = 


f  €  e*2  /°C  fV  *  £  /°v»oo 

/  /  0(C-  o,  aw)  /  4>(vi  “Q,cru)  4>(v,v0<yJdvdT)dld£ 

^ 77  =  -00  ©/  v  a  £ 

(Same  as  numerator  with  =  —  1,  *2  =  I) 


(29) 
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(I5i: 

The  order  of  Integration  ia  changed,  and  Borne  facts  regarding  <f>  and  $  (defined  in  the  Nomenclature)  are  used  to 
give 


f  f  4>(€t  C,  o'J  $  (£;  «o,  cu)U  -  <*>  (£5  «o*  «*£<*£ 


p(*i  <  *  <  *2  lu  i  *  <  »)  = 


(Same  as  numerator  with  =  -  1,  *2  <=  1) 


(30) 


or 


p(*l  <  *  < 


f  |®(*2i  £>  au)  -  *(*li  £»  au)  “0.  4l  “  $(£5  t>0  0-w)j  di 

<  »2  |w  <  *  <  t/)  *=- - - 


(Same  as  numerator  with  *|  =  -  1,  x2  =  1) 


(31) 


Equation  (31),  while  somewhat  lengthy,  is  amenable  to  numerical  integration,  converging  rapidly  to  the  correct 
value  even  {or  relatively  large  A  £  intervals.  The  {unction  is,  o{  course,  tabulated  (Re{.  1,  Table  II,  page  423). 
The  case  o{  normally  distributed  test  limits  with  an  error-free,  normally  distributed1  attribute  variable  is 
exhaustively  treated  in  Re{.  3. 

Another  example  which  is  ol  interest  is  the  {ixed  certain  limit  problem  with  a  triangularly  distributed 
attribute  variable  and  normally  distributed  error.  T.  W.  Hamilton  has  treated  this  problem  in  a  recent  practical 
application.  It  will  be  repeated  here  using  the  techniques  developed  herein.  Let 


fM) 


€  +  i,  -  l  <  £  <  o 
£  +  l,  o  <  i  <  l 

0  otherwise 


(32) 


*»  ~  sw(^>)  =  <#*>;  o,  crw) 


(33) 


Equation  (33)  may  be  inserted  in  Eq.  (16)  to  give 
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(°€=x  2 

/  fx(£)  4(1 -&0,  crJM€ 


p(*j  <  *  <  *2  “0  <  z  1  = 


£*=“r 


(152) 


(34) 


/  ^  / 
-'tf--!  «/C  =  u0 


^(t  -  o,  <rw) 


Note  that  the  £  integration  interval  in  the  denominator  has  been  taken  to  be  (-  1,  1),  since  fxL£)  =  0  outside  this 
interval.  The  formulas  involving  <fr  and  $  (see  Nomenclature)  are  employed  to  give 


p(*j  <  *  <  *2  l“0  S.  2  <  vq!  = 


/  fx(OI  <fidi  €,<rJdU£ 

"£  ■*!  »  C  *«o 

/»f»l  /"C-VQ 

/  fx(£)  £.  <rj  dtd£ 

Jf—1  o/C=B0 


(35) 


ff'*  2 
•'f  ■*! 


(36) 


/•tf  *1 

/  4<£)[*(«>io;  £.  -  ®(“o;  %)] 


Equation  (36),  with  /x(£)  explicitly  given  by  Eq.  (32),  may  be  integrated  numerically  in  specific  applications.  For 
example,  let 


i*0  -  —  0.5  = 
t>0  —  +  0.5  =  ^2 

cr  =  0.1 

XV  • 


Trapezoidal  integration  (with  -  0.25)  yields 

p(~  0.5  .<*.<+  0.5  |-  0.5  <  z  <  +  0.5)  =  0.92 

(The  error  due  to  the  use  of  trapezoidal  integration  will  be  no  larger  than  0.005.) 

The  interpretation  of  this  result  is  that  one  may  be  “92%  confident91  that  the  attribute  variable  lies  within 
the  tolerance  limits  if  the  error  standard  deviation  is  as  stated,  given  that  the  test  has  been  passed. 
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NOMENCLATURE 


x,  y,  tv,  z ,  u,  v  random  variables 

€ 9  A ,  O),  £,  7)f  v  the  corresponding  running  variables  or  variables  of  integration 

A,  B,  C,  ••••  events 

^  4<and”  =  “intersection”  (of  two  events) 
p(i4)  probability  of  event  A  occurring 
pU^B)  probability  of  both  A  and  B  occurring 

p(A  \B)  conditional  probability  of  A  on  B  =  probability  of  A  occurring  given  that  B  has  occurred 
fr.  f.  frequency  function 

~  distributed  with  fr .  /. 

••••  fr.  f.  of  *,  w,  ••••  evaluated  at  co,  ....  ;  i.e.,/^)  =  p(*  =  £)  or 

/°  *2 

p(*l  <.  *  <  *2^  =  /  /x(^)  d£,  etc. 


/ 

4|£(^>  conditional  fr.  f.  of  *  on  event  E  evaluated  at  <f;  i.e.,  /x|£(£)  =  p(x  =  g  |£),  etc. 

®r)  normal  /.  with  mean  fi  and  standard  deviation  cr,  evaluated  at 


1 

•as  -  exp 

Note  that  /c,  cr)  =  <#/*;  f ,  cr)  =  0(<f  -  0,  cr) 

=  0,  cr) 


♦(£»  O')  cumulative  normal  distribution  function  with  mean  and  standard  deviation  cr,  evaluated  at 

=/  0(TS  O')  </r=  1  -J°  fi,  cr)  dr 

f*2 

J  n,  cr)  dg  -  2J  n,  cr)  -  •  (fj,  ^u,  cr) 

(*j,  *2^  nignifies  the  closed  interval  lying  between  Jkj  and  *25  i.e.,  *  in  (*j,  x2)  signifies 
*1  <  *  <  *2 
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APPENDIX  A 

A  Short  Refresher  on  the  Properties  of  Frequency  Functions 


A  random  or  distributed  variable  is  one  which  may  take  on  any  value  within  a  specified  (finite  or  infinite) 
range.  To  any  interval  in  thiB  range  there  corresponds  a  probability  of  the  random  variable  falling  within  the 
interval.  The  fr.  f.  fx  of  a  continuous  random  variable  *  is  a  function  whose  integral  over  any  interval  (xj,  x2) 
equals  the  probability  that  x  lies  in  (xj,  x2): 


p(*l  <  *  <  *2} 


(Al) 


By  differentiating  this  expression  with  respect  to  x2  and  setting  x2  “  xl  ~  ~  00  (which  is  traditional). 


/■  (£)  =  -  [  p(~  ~  <  x  <  £)] 


(A  2) 


Since  probabilities  are  always  non-negative,  and  since  the  probability  of  any  event  which  is  certain  to  occur  (such 
as  -  ca  <  x  <  is  1,  any  fr.  f.  fx(£)  satisfies 


fx(i)  <  o 


(A3) 


Frequency  functions  of  several  variables  are  defined  as  follows:  the  joint  fr.  f.  fx  y  •••  (^f,  \f  ...),  integrated 
over  (xj,  x2),  (yj,  y2),  •••*,  gives  the  probability  that  x  lies  in  (xj,  x2)  and  y  lies  in  (yj,  y2)  and  ••••: 


/  v  r* 2  ry 2 

f»(*  i  <  *  <  <r<  y2^-)  =  /  /  —  4,  •••  (£,  •••) 

</*! 


(A4) 


If  only  certain  of  the  variables  in  the  expressions  above  ~  say  x  and  z  are  of  interest,  their  joint  /r.  f.  fx  z 
(£,  £)  is  obtained  by  integrating  4.  y,  *,  -  (£>  ^  C,  •••)  over  the  entire  range  (-  m  to  +  «»)  of  all  the 
remaining  variables: 
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oo  />  eo 

f» ,  O-I  I  •••  y>  2f  (^t  k  i$  •*•)  •••• 

o/  -oo  o'  — oo 


(155) 


(A  5) 


and 


P(*l  <  *  <  *2  <  *  <  *2 )  =  f  2  f  2  /,f  ,(£,  =  f  2  f  f  2 /,;  y>  .«'(£,  \,  C,  ..^dgdKdt,. 

"  X\  *zl  J  *\  of-ooJ*i 


<A6) 


(A  complete  discussion  of  frequency  functions  is  given  in  Chapters  3  and  4  of  Ref.  1.) 


APPENDIX  B 

The  Additive  Error  and  Fixed  Limits  Problem 


The  following  derivation  covers  the  case  wherein  the  variable  of  interest  plus  independent  additive  random 
error  Ib  subjected  to  a  test  with  fixed  limits  (Eqs.  14,  15,  and  16).  The  assumptions  are 


*  -  fxd). 


w  ~  sw^) 


and  the  answer  sought  is 


p(A  ^B) 

p(*l  <  *  <  *2  lu0  5.  *  +  w  <  =  ptd  \B)  = - 

p(B) 

as  in  Appendix  C. 

The  quantities  x  and  w  are  independent,  so  that 


(Bl) 


*>w  ^  fx(£)  gjw) 


(B2) 
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Defining 


2  =  X  +  W 


(156) 


(B3) 


and  using  the  formulas  of  Ref.  2(Sec.  22.2,  page  292)  provides 


(B4) 


Integrating  x  out  provides  the  fr.  f.  for  z: 


z  <=  x  +  w 


f~  /„<#>  gje-  £><*# 


(B5) 


Then 


p(*  1  <  *  <  *2 1 “0  <  1  <  vq) 


p(*l  <  x  <  1  z  £> 

p(«*0  <  *  < 


(B6) 


or 


p(*l  <  *  <  *2lB0  S.  x  <  VQ>  = 


I  fx(&  sjt-  o 

£«*!  °'Zmuo 

ft  m°°  ri~v0 

J  /*<£>/  6w(£-i)d£d£ 

J  f  «-*  °/ {“un 


(B7) 


as  before  (note  that  the  order  of  integration  has  been  reversed  in  the  denominator). 
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APPENDIX  C 

The  Distributed  Limits  Problem 


The  following  is  an  alternate  derivation  of  the  case  wherein  the  variable  of  interest,  uncontaminated  by 
any  additive  error,  is  subjected  to  a  test  with  distributed  limits.  This  problem  is  worked  out  (using  8  functions) 
in  Eqs.  (11),  (12),  and  (13).  What  is  given  is 

*  ~  /*(£).  u  ~  hu( 7)),  v  ~  hv(v) 

and  the  answer  required  is 


p(*j  <  x  <  *2  lu  <  *  <  «  p(A  |fi)  « 


pU^B) 

p(B) 


(Cl) 


defining  A  as  Xj  <  x  <  *2>  B  bb  u  <  x  <  v9  and  using  Eq«  (1).  Since  x9  u,  and  v  are  assumed  mutually  inde¬ 
pendent, 


x,  u,  v  ^  fx(£)  hu(rj )  hp{v) 


(C2) 


and 


p(*l  <  x  <  x2  |u  <  *  <  o) 


p(*l  <  x  <  x2r~'  ~  °°  <  u  <  x  <  v  <  oo) 

p(-  «>  <  *  <  ®oA“  *  <  B  <  X  <  V  <  oq) 


(C3) 


or 


p(«i  <  x  <  *2 1 u  <  *<0)  = 


Xf  **2  /°i?  /°v»oo 

/*<£)  /  dvdrjdg 

b  -*i  *^7?  *“»  •'v-f 


(Same  as  numerator  with  jtj  =  -  *2  =  <») 


(C4) 


n  .  io 


as  before. 
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ANALYSIS  OF  DEMAND  RATES 


Luis  F.  Nanni 
Statistical  Section 
Raritan  Arsenal 
Metuchen,  New  Jersey 

1.  This  report  presents  an  analysis  of  demand  data  and  demand  variabil¬ 
ity  of  Ordnance  secondary  items  used  in  organizational  and  field  maintenance. 


2.  Out  of  approximately  1*00,000  Ordnance  items  that  appear  in  supply 
catalogs,  about  50,000  are  considered  active  and  authorized  for  stockage  at 
field  level.  These  are  the  items  that  are  available  at  direct  support  com¬ 
panies  and  post  ordnance  installations  and  meet  the  criterion  of  having  been 
demanded  by  some  using  unit  a  minijnum  of  3  times  in  180  days.  This  criterion 
of  activity  is  currently  under  revision  so  as  to  include  other  frequencies 

of  demands  for  a  360  day  period.  If  an  item  does  not  meet  the  criterion  of 
activity  it  is  considered  a  ,,fringe,,  item  and  only  stocked  by  the  depots, 

3.  The  degree  of  activity  as  measured  by  the  quantitative  demand  on 
about  54,000  AFSCS  active  items,  as  indicated  in  Stockage  List  #8  for  Calen¬ 
dar  Year  1958  prepared  by  the  Ordnance  Supply  Analysis  Agency,  is  approxi¬ 
mately  as  follows  i 


Items  with  zero  annual  demand 

Items  with  a  frequency  of  demand  less  than 
6  times  in  360  days 

Items  with  more  than  6  demands  per  year 


$0% 

2$% 

2% 


The  relatively  high  percentage  of  items  with  either  zero  demand  or  six  or 
less  number  of  demands  per  year  may  be  due  to  the  following  reasons? 

% 

.  .  ,  a*  ,  are  considered  "stand  by"  or  insurance  type  items 

and  included  in  AFSCS  list  even  if  they  do  not  meet  the  criterion  of  activity. 

b.  Items  that  no  longer  meet  the  cri  terlon  of  activity  but  atralv 
to  end  items  still  in  the  field  and  are  being  used  to  attrition, 

c.  Items  that  pertain  to  newly  introduced  major  items* 

d.  Items  that  the  NICF  consider  superseded-by  or  exhausted-to  some 
preferred  item  and  are  deleted  from  the  AFSCS.  The  stockage  list  will  men¬ 
tion  the  preferred  item  which  may  have  very  little  demand  during  the  period, 

e.  Items  added  to  stockage  list  after  0SAA  cut-off  date  for  pro¬ 
cessing  demands  (about  three  month  interval). 

f .  Items  that  reporting  station  may  neglect  to  omit  after  it  has 
become  inactive . 


160 


Fourth  Conference  on  Operations  Research 


It  can  be  said  that  out  of  5>k>000  acti  ve  items  only  about  25$  or  13,^00 
of  them  will  have  some  degree  of  activity.  A  good  10$  of  these  items  will 
probably  have  small  quantitative  demand  which,  when  related  to  the  density 
of  the  major  items,  may  give  a  rate  very  close  to  zero.  It  thus  appears  that 
rates  will  be  computed  on  about  12,000  items,  that  is,  on  about  22$  of  the 
active  items  appearing  in  the  Amy  Field  Stock  Control  System. 

li.  The  analysis  to  be  described  in  this  research  will  refer  to  the 
22$  of  items  that  show  some  degree  of  activity  during  the  year. 

5«  The  data  needed  for  the  computation  of  demand  rates  originates 
from  the  Amy  Field  Stock  Control  System  of  recording  demands.  Three  basic 
decks  of  cards  are  needed  for  rate  computations . 

a*  Demand  data  sent  by  direct  support  organizations  from  7th 
Army  and  by  Post  Ordnance  installations  from  CONUS  and  other  theatres. 

b.  Density  of  major  item  supported  by  the  installtions.  This 
density  report  also  includes  a  tentative  classification  of  the  type  of  activ¬ 
ity  to  which  the  major  item  is  subjected. 

c.  Application  deck  sent  by  the  National  Inventory  Control 

Points. 

d.  There  are  other  decks  of  cards  which  are  used  in  the  EAM 
computational  process.  These  decks  are  needed  for  updating  the  numbers  that 
identify  the  secondary  items,  and  for  the  determination  of  items  that  become 
obsolete,  are  lost  to  another  technical  service,  or  are  superseded  by  other 
items. 


6,  The  demand  data  analyzed  in  this  research  is  the  demand  at  Post 
Ordnance  installations ,  As  such,  it  reflects  very  closely  the  actual  use  Of 
the  item  at  field  level.  An  original  demand  as  shown  in  a  "single  line  requi¬ 
sition"  (Form  151*6)  is  summarized  monthly  at  either  the  post  ordnance  instal¬ 
lation  or  the  direct  support  organization  in  an  EAM  card  and  sent  to  0SAA  for 
analysis.  Six  monthly  demands  are  available  in  each  demand  data  card.  When 
these  cards  are  shipped  to  0SAA,  an  accompanying  letter  called  the  "cover 
letter",  showing  the  volume  of  major  items  supported  during  that  period,  is 
also  sent  to  0SAA  for  review. 

7.  Approximately  111*  units,  world  wide,  will  send  above  demand  data 
information  for  analysis.  The  breakdown  of  these  units  is  as  follows?  1*5 
from  CONUS,  37  from  7th  Army,  21  from  the  rest  of  Europe,  7  from  AFFE  1* 
from  Alaska  and  other  areas.  Approximately  720,000  EAM  cards  with  demand 
information  are  received  every  year.  This  figure  includes  300,000  from  GONUS, 
170,000  from  7th  Amy  and  250,000  from  the  rest  of  the  world.  About  1*0,000 
density  cards  are  also  being  received  from  the  using  organizations.  The  vol¬ 
ume  of  cards  of  the  other  decks  needed  for  rate  computation  is  as  follows? 
application  deck  -  200,000,  "change-to"  file  -  170,000,  "transfer  and  change- 
note"  file  -  370,000. 
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8.  For  classification  purposes,  the  factors  affecting  demand  varia¬ 
bility  can  be  divided  into  two  general  categories; 

a.  Major  factors  for  which  a  weight  can  be  determined  by  means 
of  statistical  methods  on  the  basis  of  data  gathered  from  the  using  units. 

b«  Other  factors  which  will  be  categorized  as  possible  causes 
for  unexplained  fluctuations  (random  errors)  in  the  reported  data.  Analysis 
of  these  factors  will  facilitate  the  elimination  of  spurious  information  due 
to  reporting  errors  and  also  provide  a  basis  for  studying  the  causes  of  im¬ 
proper  demands  within  a  particular  area  or  unit. 

9.  Major  factors  affecting  the  demand  rate.  Three  major  factors 
are  scheduled  to  be  considered  in  the  rate  computation.  These  are:  density 
of  major  items  supported,  type  of  utilization  of  the  major  item  and  climatic 
conditions. 


a.  For  some  type  of  major  items  it  is  possible  that  either 
the  number  of  rounds  fired  (weapons)  or  miles  traveled  (wheeled  or  tracked 
equipment)  or  hours  of  operation  should  be  preferred  to  the  density  supported 
by  the  reporting  units.  No  diata  of  these  factors  is  available  on  a  regular 
basis  from  all  the  reporting  units.  Collection  of  this  type  of  data  is  a 
formidable  task  and  it  is  doubtful  that  reliable  figures  will  be  obtained 
from  the  using  units.  The  density  of  a  major  item  is  a  broad  index  of  con¬ 
sumption^  it  is  readily  available  and  can  be  easily  used  for  prediction  of 
future  demands. 


b.  The  type  of  utilization  of  the  equipment  can  be  measured 
from  information  that  may  be  available  on  maneuvers,  field  exercises  and  train¬ 
ing  programs.  Other  types  of  utilization  are:  internal  garrison  duties  and 
stand-by  status. 

c.  Knowledge  of  the  geographical  area  and  season  in  which  the 
equipment  was  used  will  determine  several  categories  of  climatic  conditions. 

It  is  possible  that,  on  final  analysis,  only  extreme  climatic  conditions  will 
affect  significantly  the  demand  rate  computed. 

10.  Analysis  of  factors  causing  random  errors. 

a.  Factors  affecting  demand  variability*  Some  of  these  factors 
will  tend  to  indicate  a  demand  from  the  field  larger  than  the  actual  use  of 
the  item;  as,  for  instance,  artificial  demands  due  to  out— of— stock  position, 
cancellations,  artificial  demands  due  to  availability  of  funds,  unauthorized 
applications.  Other  factors  may  tend  to  show  a  demand  smaller  than  the 
actual  consumption,  as,  for  instance,  a  demand  filled  by  a  unit  pack  instead 
of  its  original  number,  causing  a  lack  of  report  of  future  demands,  fringe 
nemands  from  stations  where  item  is  not  on  their  stockage  list,  artificial 
disposition  of.  stock.  Other  factors  may  tend  to  work  in  either  direction; 
for  instance,  incorrect  classification  of  recurrent  vs.  nonrecurrent  demands 0 
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However,  due  to  the  fact  that  these  demands*  are  analyzed  at  post  ordnance 
level,  the  effect  of  above  factors  is  not  considered  to  be  as  imoortant  as 
in  the  case  when  they  are  analyzed  at  depot  level.  The  only  exception  may 
be  the  "locally  fiinge"  demands  which  are  presently  not  used  in  the  computa¬ 
tion  of  demand  rates  due  to  processing  difficulties.  An  item  is  considered 
locally  fringe  when  it  is  stocked  only  at  some  of  the  reporting  stations. 

Demand  rates  based  on  "active"  demands  only,  .should  in  general  be  lower  than 
the  demand  rate  obtained  when  both  active  and  fringe  demands  are  included 
in  the  calculations.  These  fringe  demands  are  annotated  on  the  forms  151*6 
which  are  also  sent  to  OSAA  for  analysis.  A  future  report  will  show  the 
effect  of  these  demands  on  the  demand  rate. 

b.  Factors  affecting  density  variability.  The  density  sub¬ 
mitted  by  the  using  units  should  be  an  average  covering  the  period  of  the  re¬ 
port.  Density  information  is  available  on  a  semi-annual  basis  and  it  is  sus¬ 
pected  that  the  figure  quoted  is  the  amount  supported  as  to  the  date  of  re¬ 
port,  which  may  not  cover  any  density  fluctuation  during  that  period.  It  is 
also  known  that  some  units  evacuate  equipment  to  a  different  echelon  of  mainten¬ 
ance  and  continue  to  report  these  major  items  as  supported  equipment.  Errors 

of  this  type  may  be  eliminated  if  the  control  of  major  items  is  part  of  an 
EAM  card  with  correct  identification  prepunched,  and  it  is  required  that  the 
reporting  units  post  quarterly  figures  in  density  supported  and  information 
of  evacuated  equipment  and  overflow  maintenance. 

c.  Other  factors  causing  random  errors. 

(1)  Quality  of  material,  operations  and  maintenance.  This 
classification  includes  those  factors  related  to  the  quality  of  the  repair 
part  on  items  under  consideration,  to  the  quality  of  the  operator  of  the  equip¬ 
ment  and  to  the  existence  of  and  the  adherence  to  a  prescribed  maintenance 
plan  covering  the  equipment.  Specific  factors  are? 

Quality  of  items:  a.  Original  specifications  or  de¬ 
sign  not  adequate  for  application. 

b.  Use  of  sub-standard  material 
obtained  through  local  purchase 
or  manufacture. 


*  The  effect  of  these  factors  should  be  investigated  by  conducting  a 
controlled  experiment  at  field  level.  A  team  of  research  workers 
should  conduct  a  survey  at  some  randomly  selected  reporting  stations 
throughout  the  world  and  examine  the  behavior  of  the  demands  on  a 
random  sample  of  secondary  items. 
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Quality  of  operators: 


Quality  of  maintenance: 


a.  Abuse  of  equipment 

b»  Degree  of  skill  or 
training  involved 

a.  Facilities 

b.  Skill 


(2)  Reporting  errors.  This  broad  category  contains  those 
factors  caused  by  human  error  in  gathering  and  recording  data  relative  to  de¬ 
mand  and  densities.  At  best,  a  well-designed  reporting  system  can  be  expected 
only  to  minimize  errors  in  this  respect.  The  amount  of  control  necessary  to 
eliminate  errors  entirely  would  be,  in  general,  prohibitive  in  manpower  and 
costs.  When  errors  of  this  type  are  extreme  and  obvious  it  may  be  necessary 
to  withdraw  the  spurious  data  from  the  computation. 

11.  As  mentioned  above,  other  decks  of  cards  are  needed  for  the  rate 
computation.  Mention  should  be  made  of  the  condition  of  the  application 
deck.  Application  data  received  from  the  National  Inventory  Control  Points 
is  still  incomplete  and  with  many  errors.  The  original  Master  Application 
Deck  consists  of  approximately  800,000  cards.  By  eliminating  the  major  item 
applications  that  are  not  used  in  the  field,  and  by  grouping  those  major  items 
that  are  similar  in  nature,  the  Technical  Seotion  of  OSAA  was  able  to  reduce 
the  application  deck  to  about  200,000  cards. 

12.  Age  of  Equipment.  A  demand  rate  is  obtained  based  on  demand  and 
density  reported  under  different  conditions  of  age  and  usage  of  the  equip¬ 
ment  fleet.  It  is  possible  that  an  isolated  unit  may  have  either  all  new 

or  old  equipment  during  a  year,  but  it  is  more  likely  that  both  types  of  equip¬ 
ment  were  represented  in  the  average  density  of  the  majority  of  the  report¬ 
ing  units. 


The  assumption  is  made  that  the  average  age  of  the  equipment  from 
all  the  reporting  units  will  remain  approximately  the  same  from  year  to  year. 
However,  in  the  analysis  of  the  demand  rates  it  will  be  possible  to  single 
out  those  units  with  substantially  low  or  high  rates  when  these  rates  are 
due  to  new  and  old  equipment,  respectively.  Substantial  changes  of  the 
average  age  of  the  fleet  will  also  be  reflected  in  the  change  of  the  demand 
rate  which  will  be  computed  every  six  or  twelve  months. 

13.  Demand  and  density  data  for  approximately  three  years  of  data  will 
be  available  for  review  by  October  1959*  This  report  will  give  a  partial 
analysis  of  the  results  obtained  in  a  sample  of  75  items  selected  by  the 
Equipment  Specialists  of  the  Technical  Section  of  OSAA.  This  sample  was 
selected  on  the  basis  of  the  activity  of  the  item  and  should  not  be  con¬ 
sidered  as  a  random  sample  from  the  population  of  active  items. 
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The  percentage  distribution  of  the  75  items  according  to  unit  price 
and  volume  of  demand  is  as  follows: 


DEMANDS 


UNIT  PRICE  $ 

1-99 

100-999  1 

..000-9.999 

10,000 -99.999 

TOTAL 

.000  - 

.010 

0  % 

0  % 

0  % 

0  % 

0  % 

.011  - 

.100 

11 

2.3 

1*.6 

0 

8 

.101  - 

1.000 

1*.6 

9.2 

12.6 

1.2 

27.6 

1.001  - 

■  10.000 

6.9 

20.7 

9.2 

1.1 

37.9 

10.001 

-  100.000 

1*«6 

10.3 

2.3 

12 

18.1* 

100.001 

.  -  1000.00 

1.1 

1.2 

3.5 

1.1 

6.9 

1000.001  -  10,000.00 

1.2 

0 

0 

0 

1*2 

TOTAL 

19.5 

1*3.7 

32.2 

5 

100*00 

The  composition  of  the  sample  selected  from  each  group  of  major  acti¬ 
vity  is  as  follows : 


TOTAL: 


6  items 
12  items 
10  items 

9  items 
9  items 
12  items 
10  items 

7  items 

7!> 


from  Small  Arms 
from  Antiaircraft  Artillery 
from  Combat  Vehicles 
from  Non-Combat  Vehicles 
from  Electronic  Fire  Control 
from  Optical  Fire  Control 
from  Guided  Missiles 
from  Hardware 


An  additional  random  sample  of  50  items  will  be  incorporated  into  this  study, 
results  of  which  will  be  shown  in  a  future  report.  This  sample  was  selected 
by  M.I.T.  for  the  research  in  secondary  item  supply  control  for  the  Inventory 
Management  Project,  Office  of  Ordnance  Research. 

ll*.  Three  years  of  field  demand-density  data,  covering  FT  *56,  FT  *57 
and  FT  *58  was  furnished  RCA  for  the  purpose  of  computing  demand  rates.  The 
RCA  rate  contract  will  be  completed  in  September  1959.  The  demand  rates  will 
be  listed  in  two  sequences:  the  secondary  item  stock  number  sequence  fol¬ 
lowed  tjrits  applicable  major  items  and  the  major  item  stock  number  sequence 
followed  by  all  of  its  applicable  secondary  items.  .The  proposed  rate  format 
is  as  follows: 


a)  SISN  NOUN  U/M  SNL 

251*0  770  2985  gaFaS’st  ea  g5oi 

0251* 


Sg  QSg 
306  1 

3061  2 


MESN 

2320  223  8120 
2320  796  5986 


QPMI 

1 

2 


CONUS  ft»56 
RATE  ST. ERROR 

"T31  rsu 

.62  .09 


CONUS  FT' 57 
RATE  ST. ERROR 

T3B  To8 
.71*  .11 


7TH  ARMT  FX57  CONUS  FT' 58  ?TH  ARMT  FT*  58 
RATE  ST. ERROR  RATE  ST. ERROR  RATE  ST. ERROR 

.35  To?  3B  ^55  7U1' '  .09 

.71  .10  .71*  Ol  *75  .31 
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b)  MSN  NOMENCLATURE  SNL  SISN  NOUN  U/k  OPMT 

1015  59l”Ol86  MORTAR  LPT  2 IN  M2  W-E  AOTo  1015  7lJ“U391  KNOB  EA  “T“ 

CONUS  FY«56  CONUS  FY*57  7TH  ARMY  F#7  CONUS  FY>58  7TH  ARMY  FP58 
RATE  ST.  ERROR  RATE  s¥  .ERROR  RATE  ST. ERROR  RATE  ST  .ERROR  RATE' ST. ERROR 

^53  ^5  38  31  “HS  "3? — 33 - 31 — 35 — 

15.  Computation  of  the  Demand  Rate  for  a  single  application  item.  Ex¬ 
hibit  No.  1  shows  the  data,  formulas  and  charts  associated  with  the  computa¬ 
tion  of  a  single  application  rate.  It  includes* 

a.  A  list  of  the  post  ordnance  units  in  CONUS  that  reported  demand- 
density  data  for  one  year.  The  list  contains  the  following  information: 

Column  $1  is  the  unit  designators.  Column  $2.  is  the  actual  reported  yearly 
densities.  Column  $3  is  the  scaled  densities  and  Column  is  the  yearly  de¬ 
mands. 


b0  The  chart  contains  the  following  information: 

(1)  SISN 

(2)  Noun 

(3)  Major  Item  Application 

(U)  Annual  demands  -  vertical  scale 

(5)  Annual  densities  -  horizontal  scale 

(6)  Rate  Line  -  line  from  origin 

(7)  Demand-Density  Points  for  each  CONUS  unit 

(8)  Confidence  Belt  -  Broken  line  which  depicts  an  area  of 

two  standard  errors  above  and  below  the  rate  line* 

(9)  Basic  Demand  Rate  ■  b 

(10)  Demand  Rate  for  100  Major  Items  for  15  days  »  DR 

(11)  Standard  Error  ■  s 

(12)  Standard  Error  of  Estimate  »  s^ 

(13)  Standard  Error  of  Estimate  for  100  Major  Items  for 

15  days  sDR. 

(ill)  Coefficient  of  variation  ■  c 

The  demand-density  chart  furnishes  a  graphic  portrayal  of  demand- 
density  data  for  post  ordnance  units  in  CONUS  for  fiscal  year  1957  as  received 
by  Raritan  Arsenal. 

Demand-density  points  falling  within  the  confidence  belt  are 
considered  to  be  acceptable  points.  Those  points  falling  beyond  the  confi¬ 
dence  belt  are  considered  unusual  or  abnormal  points  in  that  the  rate  of 
demand  is  above  or  below  normal  in  respect  to  the  number  of  major  items  sup¬ 
ported.  The  unusual  points  in  the  chart  such  as  unit  designators  09038  and 
°5010  should  be  investigated  to  determine  the  reasons  for  the  abnormalities 
e.g.  type  of  activity,  maintenance  policies,  climatic  conditions,  etc. 
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Co  Distribution  of  Residuals0  This  chart  shows  a  frequency  distri¬ 
bution  of  residuals  in  terns  of  the  standard  error.  If  the  assumption  of 
normality  is  satisfied,  the  residuals  should  be  symmetrically  distributed 
about  zero  with  99#  of  the  residuals  falling  between  -3  and  3.  For  this 
item,  the  assumption  of  normality  is  not  unreasonable.  (See  Exhibit  1) 

16.  Computation  of  a  demand  rate  for  a  multiple  application  item. 
(Exhibit  2)  A  secondary  item  having  two  or  more  applications  is  called  a 
multiple  application.  Exhibit  §2  gives  an  example  of  a  multiple  application 
item.  The  secondary  item.  Fuse,  Cartridge  Lamp  25V  applies  to  the  following 
two  major  items:  F 3bZ  which  has  a  quantity  per  major  item  of  one  and  TOOU 
which  has  a  quantity  per  major  item  of  two.  The  densities  of  the  two  appli¬ 
cations  are  added,  after  weighing  each  application  according  to  the  quantity 
per  major  item,  and  the  rate  is  computed  following  the  techniques  of  a  sin¬ 
gle  application  item.  Thus,  and  U2  represent  the  densities  of  the  major 
item  applications,  F3l*2  and  IDOli  respective,  and  and  m2  represent  the 

quantity  per  major  items  (number  of  secondary  items  incorporated  into  the 
major  items).  The  combined  density  is  computed  according  to  the  following 
method: 


U  ®  m  U  +  in  U 
11  2  2 

This  method  assumes  that  if  a  secondary  item  is  used  in  two 
or  more  different  types  of  major  items,  the  rate  of  use  will  be  linearly  re¬ 
lated  to  the  quantity  of  secondary  item  incorporated  in  each  of  the  major 
items.  Variations  due  to  different  quantities  (m)  in  which  the  secondary 
item  applies  to  each  major  item  are  properly  accounted  by  weighing  each  den¬ 
sity  by  the  proper  value  of  m  before  obtaining  the  combined  density  (U) . 
Thus,  the  Fuse,  Cartridge  Lamp  2£V  applies  one  (l)  time  in  an  F3l|2  and  two 
(2)  times  in  a  IDOl*.  Unit  designator  0 lOltf  (Exhibit  #2)  reports  a  yearly 
demand  of  ll+h  fuses  and  a  density  of  five  (5)  F3h2  and  twelve  (12)  YOOli, 
then  the  assumption  is  that  the  liih  fuses  were  used  in  a  combined  density 
(1)  (5)  +  2(12)  =29.  That  is,  the  lltli  fuses  were  used  in  29  different 
locations. 


The  demand  rate  obtained  after  the  densities  are  combined  repre¬ 
sents  the  demands  for  100  locations  for  15  days.  To  obtain  the  demand  rate 
for  100  major  items,  the  demand  rate  per  location  is  multiplied  by  the  Quan¬ 
tity  per  major  item.  *  4 

17 .  Using  the  field  demands  for  FI  1957,  the  frequency  distribution  of 
number  of  applications  is  as  follows: 


Number  of  applications:  1  32.1# 

2  19*6# 

3  10.3# 

h  3ok% 

5  lt.Q# 

6  5.0# 

7  6.0# 

8  1.6# 

9  1.6# 

10  5«6# 

more  than  10  10.8# 
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18.  Analysis  of  Rates.  A  complete  analysis  of  the  demand  rates  calcu¬ 
lated  for  FY  19^6,  1957  and  1958  will  be  shown  in  Statistical  Progress  Report 
No.  8  to  be  published  in  the  near  future.  We  should  now  briefly  summarize 
the  results  obtained  using  the  sample  of  75  items  from  demand-density  data 
from  CONUS  for  FI  1957. 

a.  Coefficient  of  Variance 


(1)  The  coefficient  of  variance  (c)  is  defined  as  the  ratio 
of  the  standard  of  estimate  error  to  the  demand  rate.  The  distribution  is 
as  follows: 


_c _ 

Related  Frequency 

o-U 

0$ 

5-9 

16$ 

10-1U 

28$ 

15-19 

11$ 

20-21; 

11$ 

25-29 

9% 

30-3U 

7$ 

3 5-39 

2$ 

ko-Uk 

2$ 

IS- 80 

H$ 

(2)  It  is  seen  that  of  the  items  have  a  c  less  than  15$, 

31$  have  a  c  between  1$%  and  305?  and  25$  have  a  c  of  more  than  30$. 

(3)  Items  with  a  coefficient  of  variance  of  less  than  15$  are 
considered  statistically  reliable  f  Figures  1  through  8  show  examples  of 
those  items.  If  points  with  residuals  of  +  2  standard  errors  (points  out¬ 
side  the  confidence  belt)  are  eliminated  from  the  calculation,  51$  of  the 
items  in  this  group  will  have  their  rate  changed  by  +  1C %  and  about  915?  of 
the  items  will  have  their  rate  changed  by  +  205?.  The  average  change  in  the 
rate,  after  elimination,  is  —125? .  It  appears  that  the  demand  rates  for 
secondary  items  with  c  less  than  155?  are  not  changed  substantially  by  the 
elimination  of  points  outside  the  confidence  belt. 

(it)  Items  with  a  coefficient  of  variance  between  155?  and  305? 
should  be  closely  examined  for  possible  explanation  of  unusual  points. 

Figures  9  thru  16  show  examples  of  those  items.  The  average  change  in  the 
demand  rate  after  elimination  of  unusual  points  is-33$. 

(5)  Figures  17  thru  2h  show  examples  of  items  with  a  coeffi¬ 
cient  of  variance  of  more  than  30$.  Items  in  this  group  should  be  thoroughly 
examined  for  a  statistical  and  technical  review.  The  following  courses  of 
action  are  suggested: 

(a)  Improvement  may  be  made  in  the  demand  rate  by  elimin¬ 
ation  of  unusual  points.  An  example  in  this  group  is  the  rate  for  a  connector 
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used  in  G2 5k,  G2 $6  and  G287.  The  rate  before  and  after  elimination  of 
unusual  points  is  102  and  28,  respectively,  for  100  major  items  and  l£ 
days.  The  reason  for  the  high  rate  was  an  annual  demand  of  ljj.,000  connectors 
from  Fort  Benning,  an  active  training  center. 

(b)  There  is  unsufficient  demand  data  for  the  rate  com¬ 
putation.  By  combining  two  years  of  data  it  may  be  possible  to  improve  the 
rate.  This  may  be  done  for  items  in  this  group.  The  new  rate  obtained 
smaller  coefficient  of  variance.  (See  P.R.  #8) 

(c)  Other  factors  beside  density  are  affecting  the  demand. 
Technical  analysis  of  these  rates  should  give  an  indication  of  the  existence 
of  these  factors. 

b*  Reduction  of  demand  variability 

As  stated  previously,  the  density  of  major  items  is  only 
one  of  the  factors  affecting  demand  variability.  Eliminating  the  effect  of 
densities. in  demand  variability,  an  estimate  can  be  obtained  of  the  varia¬ 
bility  due  to  other  factors.  It  is  easy  to  show  that? 

E(di-d)2  ■  E(di4>Ui)2  -  dE(di-bUi)  +  E(W-d)2 


The  variability  of  the  demands  with  respect  to  themselves 
is  measured  by  the  quantity  on  the  left  hand  side.  The  first  two  terns  in 
the  right  hand  side  measure  roughly  the  variability  of  the  demands  with  re¬ 
spect  to  the  line  of  best  fit.  The  last  term  measures  roughly  the  variability 
explained  by  the  line  of  best  fit.  The  ratio  of  this  latter  variability 
to  the  variability  of  the  demands  will  be  called  the  percentage  reduction 
of  variability  of  demands  due  to  the  least  square  fit.  A  high  value  of  this 
percentage  will  indicate  a  good  fit,  that  is,  most  of  the  demand  variability 
is  due  to  changes  in  density. 


Items  with  c  <15$  2$  <  c  <,30$  c>  30$ 

Average  percentage 

Reduction  of  Variability  70$  1^0%  10$ 

It  is  seen  that  for  items  with  low  coefficient  of  variance 
most  of  the  variability  of  demand  is  due  to  changes  in  density. 

c*  Time  variability  of  demands 


By  computing  the  demand  rate  of  items  for  successive  periods 
of  time  it  will  be  possible  to  classify  the  items  in  four  categories? 

(l)  Items  whose  demand  rates  do  not  change  in  time.  Fluctua¬ 
tions  observed  in  the  demand  rate  will  be  called  normal  if  they  are  within 
two  average  standard  errors. 
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(2)  Items  whose  demand  rates  increase  with  time 

(3)  Items  whose  demand  rates  decrease  with  time 

(h)  Items  with  fluctuating  demand  rates 

For  the  sample  of  items  considered  the  percentages,  1*6$, 
18$, 1 8$  and  18$,  were  observed  in  above  categories. 

Demand  rates  for  items  with  stable  rates  and  with  a  coeffi¬ 
cient  of  variation  of  less  than  10$  are  ideal  predictors  of  future  demand. 

Above  information  will  be  available  for  all  active  items 
when  several  years  of  rates  have  been  computed. 

d.  Prediction  of  demands 


Demand  rates  measure  past  utilization  of  an  item  in  relation 
to  the  density  of  major  items.  If  conditions  other  than  density  change  sub¬ 
stantially  from  year  to  year,  a  demand  rate,  no  matter  how  good,  will  not  be 
a  good  predictor  of  future  demands. 


ing  to: 

DS7 


(l)  Prediction  of  field  demands  using  field  rates 

A  predicted  demand  for  the  year  1957  was  formed  accord- 

bS6  d57 


where  is  the  basic  rate  for  1956  and  Ugj  is  the  density  for  1957*  This 

predicted  demand  was  compared  with  the  actual  demand  and  the  following  ratio 
was  obtained: 


-BS7  (active) 

Dtjy  (predicted) 

For  stable  items  with  c  15$  the  median  value  of  P  was 
-.05  and  the  range  was  between  -.9  and  .6.  For  all  items,  the  median  was 
0.13  and  the  range  was  between  -3.9  and  .9.  More  complete  information  on 
this  type  of  prediction  will  be  given  in  Progress  Report  #8. 

(2)  Prediction  of  all  demands 

A  prediction  of  world  wide  demands  was  attempted 
using  a  field  demand  rate  and  a  projected  world  wide  density.  For  purposes 
of  comparison,  information  from  the  NICP  was  requested  on  their  prediction 
of  future  requirements  and  the  actual  issues.  In  1+8$  of  the  cases  both 
predictions  were  under  and  in  19$  of  the  cases  both  were  over  the  actual 
issues.  In  general,  it  can  be  said  that  predictions  of  issues  based  on 
field  demand  rates  are  lower  than  prediction  of  issues  based  on  NICP  methods 
of  predicting. 


„  °S7  (predicted) 

P  M  - 
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OS AA  rates  are  field  rates  and  should  be  used  to  predict 
only  field  demand.  In  order  to  obtain  a  prediction  of  all  the  demands,  to 
the  field  demands  based  on  field  rates,  the  following  demands  should  be  added: 
rebuild  demands  based  on  rebuild  rates  and  extra  demands  generated  at  depot 
level.  It  is  hoped  that  Progress  Report  #8  will  throw  some  light  into  this 
method  of  prediction. 

e.  Use  of  a  Poisson  Model  for  Rate  Computation 

If  demands  behave  in  a  Poisson  fashion  then  the  square 
root  of  demands  should  fit  linearly  to  the  square  root  of  the  density.# 

A  fit  of  this  type  was  made  for  approximately  60  items  and 
a  comparison  was  made  with  a  fit  using  demands  versus  density.  Tho  coefficient 
of  variance  for  the  first  fit  is  generally  lower  than  the  coefficient  of  vari¬ 
ance  for  the  second  fit.  However,  when  compared  with  future  demands,  the 
"square  root  fit"  underpredicts  even  more  than  the  demand  versus  density  fit. 
(See  Progress  Report  #8) 

f  •  Future  Plans 

(1)  Use  of  residuals  to  improve  prediction 

I'd  improve  predictions,’  'the  following  method  is  being 
tested:  =  =*S6  U57  +  %6  r56 

where  a^  and  c^  are  coefficients  obtained  by  least  squares  using  1956  data, 

U57  ls  the  projected  density  for  1957  and  r^  are  the  residuals  left  unexplained 

from  1956  fit.  Some  improvements  had  been  observed  in  this  method  of  predic¬ 
tion.  More  results  will  be  shown  in  Frogress  Report  #8. 

(2)  Model  accounting  for  a  possible  variation  of  the 
standard  error  in  terns  or  the  density 

"  -»«“-““•»>  The  formulas  illustrated  in  Exhibits  1  and  2  are  based 

on  a  constant  standard  error.  It  if  possible  to  account  for  a  change  of  stan¬ 
dard  error  in  terms  of  the  density.  Two  models  are  being  tested:  a)  standard 
error  changes  linearly  with  the  density]  b)  standard  error  changes  according 
to  the  square  root  of  the  density.  In  the  first  case  the  basic  rate  is: 

(a)  b  -  C 

Ui 

and  in  the  second  case  the  basic  rate  is: 

(b>  b2  -  s  diui 
ui 


#  Suggestion  made  by  Dr.  J.  Tukey  of  Princeton  University 
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When  b-^  and  b2  are  compared  with  the  b  obtained  on  the  assumption  of  constant 
standard  error, 

(c)  b  *  Ed.u. 

11 


it  is  seen  that  the  basic  demand  rate  is  an  average  value  for  the  rate  of 
each  reporting  unit  with  the  following  weights  used  for  each  reported  demands 

2 

for  b  weight  U 
for  b^  weight  U 
for  b2  weight  1 

In  fact,  the  three  formulas  can  be  written  explicitly  as  follows ; 


Eu.2 


i 

It  is  thus  seen  that  a  rate  based  on  d  versus  U  with  constant  standard  error 
is  affected  more  than  the  other  rates  by  points  with  high  demand  at  high  den¬ 
sity  or  low  demands  at  low  densities. 

(3)  Effect  of  fringe  demands  in  the  demand  rate 
,  _  Some  demand  rates  are  quite  low  due  to  the  effect  of  very 
low  demand  at  high  density.  It  is  possible  that  some  of  these  demands  were 
not  accounted  for,  due  to  their  being  "locally  fringe  demands".  The  effect 
of  these  demands  are  presently  under  consideration.  By  incorporating  all  the 
demands  reported  in  1 BU6  Forms  in  the  computation  of  rates,  it  will  be  pos¬ 
sible  to  determine  the  importance  of  these  extra  demands.  The  task  is  very 
difficult  because  of  the  large  number  of  forms  that  have  to  be  researched 
and  because  these  demands  are  not  in  IBM  punched  card  form. 

19.  Technical  Analysis  of  Rates.  The  most  important  function  of  OSAA 
is  the  technical  analysis  of  demand  rates  for  maintenance  and  procurement 
purposes.-  The  ultimate  purpose  of  this  research  Is  to  attempt  a  classifica¬ 
tion  of  the  secondary  items  in  families,  using  perhaps  dollar  value  as  a 
yardstick,  according  to  the  categories  found  to  be  of  significance  in  demand 
variability.  A  family  may  include  items  that  are  interchangeable,  exhausted 
to  or  from  another  item.  Considerations  of  the  material  of  which  the  part 
is  made,  function  that  the  part  performs,  location  of  the  part  in  the  major 
item,  and  policy  about  mandatory  replacement  may  play  an  important  role  in 
this  classification. 
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A  COMPUTER  SIMULATION  OF  SMALL  COMBAT  ACTIONS 


John  C.  Flannagan 
Combat  Operations  Research  Group 
Combat  Developments  Section,  Hq,  USCONARC 

The  AUTOTAG  computer  simulation  has  been  designed  to  evaluate  antiarmor 
/  weapons  and  weapon  system  ^performance.  This  simulation  provides  the  means  to 
accomplish  a  multivariable  systems  comparison  in  a  relatively  short  time. 

This  paper  will  describe  the  AUTO  TAG 'program  in  some  detail  and  discuss 
a  few  of  the  problem  areas,  related  to  antitank  weapon  systems,  where  this 
simulation  may  find  application.  A  short  discussion  of  a  tentative  plan  for 
initial  use  of  the  simulation  will  also  be  discussed, 

BACKGROUND.  I  would  like  to  preface  this  discussion  by  providing  a  few 
background  details  related  to  simulation  development. 

About  five  years  ago,  a  large  unit  war  game  was  developed  at  CORG.  This 
game,  called  SINT  AC,  involves  combat  between  large  size  forces  and  requires 
extensive  military  player  participation.  Umpires  serve  as  the  game  control 
function  by  deciding  on  action  outcome  throughout  each  game.  SYNTAC  continues 
to  provide  information  of  tactical  and  logistical  value  related  to  large  scale 
combat,  however,  this  war  game  does  not,  in  general,  provide  detailed  informa¬ 
tion  related  to  small  unit  combat  actions. 

Consequently,  in  order  to  investigate  particular  aspects  of  armor  combat 
in  greater  detail,  shortly  after  the  development  of  SYNTAC,  effort  was  devoted 
to  the  design  of  a  small  unit  war  game,  TABWAG  (Tank  Battle  War  Game). 

TABWAG  was  specifically  developed  to  investigate  the  antitank  effectiveness 
of  weapon  systems.  Play  took  place  on  a  large  scale  terrain  map  with  players 
representing  each  force.  A  single  umpire  was  required  to  act  as  the  decision 
maker. 

TABWAG  was  in  fact  useful  for  small  unit  system  evaluation,  however, 
one  of  the  primary  limitations  of  this  game  was  that  playing  time  for  a  single 
play  required  about  25  man  hours.  Such  a  limitation  seriously  curtailed  the 
number  of  situations  that  could  be  investigated  using  this  model. 

In  order  to  reduce  playing  time  and  eliminate  the  need  for  player-umpire 
participation,  a  rigid  war  game,  TAG(rigid  in  the  sense  that  decision  criteria 
and  rules. of  play  were  built  into  the  model),  was  subsequently  designed.  The 
stylized  structure  of  TAG  made  it  possible  for  an  individual  unversed  in  mili¬ 
tary  doctrine  to  play  through  a  combat  situation. 

The  TAG  game,  or  more  precisely,  simulation,  was  used  in  a  comparative 
antitank  weapon  evaluation  where  a  factorial  design  of  five  variables  at  two 
levels  each  was  employed.  Fifteen  replications  of  each  of  the  32  combinations 
were  played  and  non-parametric  methods  of  analysis,  as  well  as  analysis  of 
variance  used  on  several  measures  of  system  performance,  A  more  complete 
write-up  of  the  results  of  this  study  as  well  as  a  discussion  of  the  TAG  model 
may  be  found  in  CORG  Report  R-71. 
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The  play  of  a  single  TAG  game  required  about  one  half  a  man  hour#  This 
was  of  course  a  distinct,  improvement  timewise  over  TABWAG,  however,  the  t-i^ 
required  for  play  of  a  single  game  was  still  excessive  for  hand  play  of  larger 
system  comparisons.  Unfortunately,  the  limited  scope  of  the  TAG  analysis 
pointed  out  a  need  for  larger  designs  involving  more  variables. 

It  was  apparent  that  in  order  to  broaden  the  scope  of  future  analysis, 
playing  time  must  be  further  reduced.  It  was  also  obvious  that  the  only 
feasible  way  to  accomplish  this  time  reduction  was  to  goto  automation. 

* 

AUTOTAG  has  been  the  next  step  in  the  developmental  cycle.  The  AUTOTAG 
program  is  an  automated  and  more  complex  counterpart  of  TAG.  The  computer 
model  was  developed  and  subsequently  programmed  for  the  IBM  70i|  at  the  National 
Bureau  of  Standards  in  Washington,  D.  C. 

AUTOTAG  PROGRAM.  The  AUTOTAG  program  is  an  internally  stored  program 
composed. of  some  5000  instructions  and  data  words.  The  structure  of  the 
program  is  such  that  it  may  be  subdivided  into  four  parts  for  ease  of  dis¬ 
cussion.  These  are:  Input,  Simulation,  Output,  and  Analysis.  I  would  like 
to  discuss  each  of  these  parts  briefly. 

Inputs 


AUTOTAG  inputs  may  be  subdivided  into  four  basic  categories  as  shown 
in  Figure  1:  _ 


1.  Weapon  type  parameters 

2.  Weapon  environmental  parameters 

3.  Weapon  change  parameters 

4.  Target  Selection  rules 

Figure  1. 

The  first  of  these,  weapon  type  parameters,  define  the  weapon  in  terns 
of  inherent  capabilities  and  limitations.  Examples  of  the  parameters  that 
must  be  specified  for  each  weapon  class  or  type  are  illustrated  in  Figure  2. 
AUTOTAG  has  the  capacity  to  handle  10  weapon  types,  and  these  parameters 
shown  would  be  input  for  each  weapon  type  to  be  gamed.  Horizontal  and  verti¬ 
cal  dispersions,  are  range  dependent,  kill  probabilities  are  based  on  weapon- 
arget  combinations,  and  performance  times  exemplified  by  load  or  obscuration 
time  are  stored  as  cumulative  distributions,  and  are  in  turn  randomly  sampled 
when  performance  time  values  are  required. 
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WEAPON  TYPE  PARAMETERS 

•  Horizontal  and  Vertical  Dispersions 

•  Conditional  Kill  Probabilities 
,  Maximum  and  Minimum  Ranges 

•  Dimensions  as  a  Target 

•  Position  Disclosure  Characteristics 
.  Performance  Times 


Figure  2. 

The  second  category  of  inputs  is  comprised  of  parameters  which  serve  to 
describe  the  status  and  role  of  each  weapon  participating  in  the  simulation. 
Examples  of  these  parameters  are  shown  in  Figure  3. 


WEAPON  ENVIRONMENTAL  PARAMETERS 

•  Side 

•  Type 

•  Initial  Position 

•  Velocity 

.  Status  as  a  Weapon 
.  Status  as  a  Target 
»  Target  Selection  Rules 

•  Field  of  Fire 

•  Artillery  and  Mine  Vulnerability 


Figure  3.  . 

Up  to  31  weapons  on  either  side  may  be  considered.  For  each  of  these 
weapons  an  initial  position  and  velocity  are  specified  with  respect  to  a  stored 
grid  network.  Weapon  and  target  status  as  well  as  field  of  fire  are  used  to 
introduce  terrain  effects.  As  examples,  terrain  masking  can  be  depicted  in 
terms  of  a  field  of  fire  limitation,  or  an  unavailable  target  status  for  a 
weapon  would  correspond  to  the  weapon  being  completely  concealed  from  an 
members  of  the  opposing  force. 

The  third  category  of  inputs,  weapon  change  parameters,  permits  modifica¬ 
tion  or  change  of  any  parameter  shown  here,  during  the  course  of  play.  The 
use  of  such  changes  can  provide  a  change  in  a  weapon's  position,  a  change  in 
a  weapon's  rate  of  advance,  or  a  change  in  a  weapon's  target  selection  rules 
as  examples.  Up  to  128  of  these  environmental  changes  may  be  executed  at 
specified  times  during  the  simulation. 

The  fourth  category  of  inputs,  target  selection  rules,  are  used  to  de¬ 
scribe  a  weapon's  fire  doctrine.  A  primary  and  secondary  target  selection 
rule  index  may  be  prescribed  for  each  weapon  —  each  index  in  turn  refers 
to  one  of  128  selection  rules  which  comprise  the  fourth  category  of  inputs. 
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Although  the  discussion  of  inputs  has  by  no  means  been  complete,  it 
should  be  apparent  from  the  illustrative  examples  that  a  variety  of  weapon 
systems  and  system  environments  may  be  defined. 

Simulation 

ATTm«mf/-iW(i)UiC*  to  briefly  indicate,.,  in  a  rather  elementary  fashion,  how 
AUTOTAG  interprets  input  data.  * 

Each  weapon  participating  in  the  simulation  can  be  in  one  of  several 
actionable  states,  such  ast  looking  for  a  target,  preparing  to  fire,  firing, 
or  obscured.  The  current  and  future  action  times  associated  with  each  weapoh 
ar®  stored  in  an  action  file.  Game  or  simulation  time  is  the  control  function, 
and  1  might  add,  also  defines  the  length  of  any  AUTO  TAG  game.  Program  control 
sorts  through  the  action  file  and  attempts  to  select  any  action  time  which 
is  equal  to  current  game  time.  If  no  action  is  found,  game  time  is  advanced 
by  one  unit  and  program  control  resorts  over  all  action  times.  Assume  that 

]re4ax?'0n  associated  w*-1*  a  selected  time  is  a  LOOK.  A  LOOK  action  inrolies 
that  the  weapon  is  prepared  to  acquire  a  target.  If  the  target  is  available 
and  may  be  selected  on  the  basis  of  parameters  prescribed  by  weapon  environ¬ 
mental  inputs,  more  particularly,  weapon  target  selection  rules,  field  of 
fire  and  weapon  target  status,  such  a  selection  occurs.  A  fire  time  is  esta¬ 
blished  for  the  weapon  on  the  basis  of  the  weapon's  performance  time  distri¬ 
bution.  This  time  is  then  stored  in  the  action  file.  The  program  resorts 
over  all  action  times  and  selects  the  next  action  time  in  chronological  order. 
Eventually,  the  lire  time  which  has  been  established  for  the  weapon  will  be 
selected  by  program  control,  unless  of  course  an  intervening  action  has  de¬ 
stroyed  the  weapon.  Assuming  such  an  intervening  action  did  not  occur,  the 
fire  action  win  be  performed.  A  computation  of  the  kill  probability  assoc- 
a  with  the  fire  action  occurs  based  on  weapon  type  inputs  such  as  weapon 
dispersions  as  a  function  of  weapon-target  range,  target  size  and  conditional 
kill  probability.  The  result  of  the  action  is  stored,  the  weapon  goes  through 
an  obscuration  phase,  and  at  a  subsequent  action  time  is  prepared  to  attempt 
to  acquire  the  same  or  an  alternate  target  for  a  new  fire  action. 

i  Performance  times  associated  with  weapon  actions  such  as  load  time,  and 
ay  and  fire  time,  p’e  established  by  random  sampling  from  appropriate  perfoxm- 
ance  time  distnbutions.  Fire  action  outcome  is  determined  by  using  a  random 
number  generation  technique  and  testing  the  generated  number  against  the  com- 
pu*bea  kill  probability  value© 

I  would  like  to  touch  on  one  additional  simulation  capability  before 
closing  this  portion  of  the  discussion.  The  AUTOTAG  program  is  able  to  simu- 
late  artillery  and  mine  action.  This  is  accomplished  through  the  specifica¬ 
tion  of  artillery  or  mine  zones.  Each  zone  is  identified  in  terns  of  square 
area  and  rate  of  artillery  concentration  or  number  of  mines  in  the  zone. 

en  a  weapon  enters  an  artillery  or  mine  zone,  a  stored  weapon  change  para- 
meter  indicates  this  fact  to  program  control.  This,  in  turn,  initiates  an 
artillery  or  mine  zone,  computation.  This  determines  if  the  weapon  sustains 
a  death  in  the  respective  zone  prior  to  maximum  game  time  (which  defines  the 
end  of  the  game;.  If  such  a  death  occurs,  the  death  action  is  stored  in  the 
action  file  and  will  occur  at  the  computed  time.  If,  however,  the  weapon 
leaves  the  zone,  m  the  case  of  artillery,  or  if  the  weapon  changes  velocity 
or  leaves  the  zone,  in  the  case  of  mines,  at  a  game  time  prior  to  computed 
time  of  weapon  death,  the  action  will  be  deleted. 
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Output 

Output  from  the  AUTOTAG  simulation  is  in  the  form  of  an  action  by  action 
record  of  all  important  events  which  occurred  during  the  simulation.  This 
record  or  log  provides  a  detailed  chronological  listing  of  action  entries , 
and  indicates  the  weapon  and  target  involved  in  each  case.  The  results  are 
in  terms  which  require  little  or  no  translation  into  realistic  measures  of 
effectiveness,  for  example:  rounds  fired,  kills  inflicted,  or  deaths  sustained. 

Analysis 

During  the  programming  phase  of  the  AUTOTAG  project,  a  good  deal  of 
effort  was  devoted  to  the  development  of  a  data  processing  routing  which  would 
simplify  a  portion  of  the  statistical  problems  associated  with  AUTOTAG  out¬ 
put.  This  effort  resulted  in  an  analysis  program  which  can  count  various 
log  entries  such  as  rounds  fired  by  a  weapon  system.  These  counts  are  auto¬ 
matically  compiled  for  all  replays  of  a  given  situation  and  then  average  values 
are  computed  together  with  the  variance  associated  with  these  average  values. 
Additional  statistical  processing  of  output  data,  for  example,  analysis  of 
variance,  has  already  been  programmed  and  will  be  accomplished  on  a  smaller 
computer. 

Problem  Areas 


I  believe  it  would  be  worthwhile  to  indicate  a  few  of  the  problem  areas 
where  it  is  anticipated  that  this  model  will  contribute  significantly  in  its 
initial  applicatiorl. 


PROBLEM  AREAS 

.  Composition  of  Antitank  systems 
.  Parameters  specified  in  QMR's  and  MC's 
.  Influence  of  weapon  parameters  on  tactics 
.  Factors  for  war  gaming 
.  Improvement  of  evaluation  techniques 


Figure  4 

As  Figure  4  indicates,  the  first  problem  area  involves  composition  of 
weapon  systems.  For  example,  what  combination  of  conventional  and  missile 
type  weapons  will  provide  optimal  combat  effectiveness?  Problem  two:  What 
can  we  say  about  parameters  specified  in  qualitative  materiel  requirements 
or  military  characteristics?  What  is  the  relative  importance  of  and  the 
relative  trade  off  among  these  parameters?  Problem  three:  What  are  the  in¬ 
fluence  of  weapon  parameters  on  tactics,  deployment  and  other  factors  associ¬ 
ated  with  combat?  Is  it  possible  that  through  a  comprehensive  analysis  of 
the  effect  of  these  factors  we  will  be  able  to  more  adequately  determine 
system  requirements?  Problem  four:  What  are  appropriate  factors  for  war 
gaming? 
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Can  we  improve  war  gaming  methods  through  a  detailed  study  of  factor  impor¬ 
tance  and  factor  effect?  Problem  five:  Finally,  how  can  we  improve  current 
simulation  and  analytical  evaluation  technique?  What  effect  will  AUTOTAG 
output  have  on  the  design  of  subsequent  simulations  or  mathematical  models? 

Although  this  is  by  no  means  an  exhaustive  listing  of  problem  area?,  it 
is  sufficient  to  illustrate  the  type  of  problems  associated  with  weapons  sys¬ 
tem  evaluation  that  currently  exist, 

TENTATIVE  PLAN  OF  EVALUATION,  The  initial  exploitation  of  the  AUTOTAG 
model  should  encompass  as  many  of  these  problem  areas  as  possible,  and  since 
a  multivariable  evaluation  is  desired,  economic  criteria  dictate  the  selection 
of  some  systematic  method  of  problem  attack,  A  rather  thorough  consideration 
of  our  speoific  needs  has  led  us  to  tentatively  select  a  factorial  design 
approach.  The  design  that  has  been  chosen  involves  twelve  factors,  or  vari- 
a^es>  a<t  'bwo  levels.  Figure  5  indicates  the  variables  which  might  be  con¬ 
sidered  in  this  design. 

,  Quantity  of  primary  weapon 

.  Type  of  primary  weapon 

.  Quantity  of  support 

•  Type  of  support 

•  Terrain 

•  Attacker  tactic 

.  Quantity  Attacker 

•  Attacker  movement 

,  Defender  deployment 

•  Range 

,  Defender  fire  plan 

.  Visibility 

Figure  5. 

As  I  have  indicated,  each  variable  will  have  two  levels;  for  example, 
one  level  of  range  may  be  600  meters  and  the  second  level,  1200  meters. 

An  effort  will  be  made  to  choose  variable  levels  such  that  a  pronounced 
difference  in  level  values  will  exist.  If  this  is  not  done,  variables  that 
are  highly  significant  in  terms  of  main  effect  may  not  be  detected  because 
of  inherent  simulation  statistical  fluctuation. 
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A  complete  two  to  the  twelfth  factorial  design  would  involve  about  1|100 
different  situations,  and  if  each  situation  is  to  be  replayed  20  to  30  tines 
to  obtain  validity,  the  cost  of  carrying  out  such  a  design  would  be  prohibi¬ 
tive.  It  is  possible,  however,  to  fractionate  a  factorial  design  substantially 
and  still  obtain  data  of  value.  *  In  the  case  of  the  two  to  the  twelfth  fac¬ 
torial  design,  fractionation  will  result  in  some  data  loss,  however,  it  will 
still  be  possible  to  measure  the  main  effect  of  each  variable  as  well  as  90% 
of  all  two  factor  interactions.  The  remaining  two  factor  and  higher  order 
interactions  are  not  measurable.  It  should  also  be  pointed  out  that  the  measur¬ 
able  effects  are  not  as  "pure”  as  we  would  like  in  that  they  are  actually 
confounded  with  higher  order  interactions. 

Output  data  from  the  design  will  be  handled  with  a  variety  of  statistical 
treatments  including  conventional  analysis  of  variance  technique  and,  per¬ 
haps,  factor  analysis. 

Subsequent  designs  may  consider  variables  at  three  levels.  For  example, 
it  is  anticipated  that  a  1/27  fraction  of  a  7  variable,  3  level,  factorial 
design  will  be  programmed,  in  order  to  learn  more  about  variables  which  had 
a  strong  effect  in  the  initial  two  to  the  twelfth  design.  In  this  instance, 
where  three  levels  of  a  variable  are  considered,  other  statistical  treat¬ 
ments  such  as  regression  analysis  or  factor  analysis  may  be  used  in  conjunc¬ 
tion  with  analysis  of  variance  technique. 

SUMMARY.  In  summary,  I  would  like  to  stress  a  few  important  points. 

AUTOTAG  is  a  highly  versatile  program  capable  of  reproducing  a  variety  of 
combat  situations.  Any  plan  for  program  utilization  must  be  oriented  to¬ 
wards  pertinent  and  timely  problem  areas  and  provide  a  sound  basis  for 
subsequent  exploitation.  It  is  felt  that  this  tentative  plan  for  exploita¬ 
tion  fulfills  these  criteria  and  provides  an  excellent  balance  between  po¬ 
tential  results  and  capital  expenditure. 


*  pie  National  Bureau  of  Standards  has  done  extensive  work  on  this  sub¬ 
ject  and  has  published  an  atlas  which  details  optimized  procedures 
for  fractionating 'two  level  factorial  designs. 


ARM!  LOGISTICS  RESEARCH 


Lt.  Colonel  H.  W.  Rice 
Research  Section,  Plans  Division, 

Deputy  Chief  of  Staff  for  Logistics 

The  purpose  of  my  discussion  is  to  acquaint  you  with  the  Army’s  logi. sties 
operations  research  activities.  I  will  cover  the  role  which  operations  re¬ 
search  plays  in  logistic  activities,  and  will  give  a  resume  of  the  logistic 
operations  research  program  objective?,  policies,  organizations,  problem 
areas  and  future  study  programs. 

What  is  logistics?  We  use  the  term  loosely.  It  is  as  old  as  organized 
warfare.  It  is  both  an  art  and  a  science.  Past  military  operations  have 
demonstrated  that  without  adequate  consideration  for  logistics,  it  spells 
the  difference  between  victory  and  defeat.  It  is  concerned  with  the  establish¬ 
ment  and  support  of  military  forces.  It  encompasses  the  harnessing  of  techno¬ 
logy  and  production  for  military  needs,  and  it  provides  for  the  proper  dis¬ 
tribution  of  the  materiel  means  of  warfare  to  the  combatant  forces. 

Today  the  gross  materiel  requirements  for  military  forces  has  put  such 
a  demand  on  the  nations  economy,  that  their  magnitude  is  controlled  largely 
by  the  impact  upon  the  economy,  rather  than  the  nature  of  the  enemy  threat. 

Consequently,  in  its  broadest  sense,  logistics  management  today  is  1) 
the  shaping  of  the  optimum  combinations  of  military  support  forces  and  equip¬ 
ment  essential  to  U.  S,  protection,  but  consistent  with  economic  stability, 
and  2)  the  distribution  of  the  proper  support  equipments  and  forces  to  the 
right  places. 

When  viewed  in  terms  of  these  requirements,  and  when  placed  in  its 
perspective  of  actions  necessary  to  determine,  create,  produce,  distribute, 
and  efficiently  utilize  the  end  products,  logistics  in  a  field  so  complex 
and  important  as  to  demand  the  application  of  the  most  sophisticated  means 
of  assistance. 

Turning  now  to  operations  research,  what  is  operations  research?  In 
simple  terms,  operations  research  is  a  tool  which  can  be  used  by  military 
management  to  assist  in  solving  its  problems.  It  consists  of  a  systematic 
approach  which  seeks  to  define  a  problem,  draws  hypothesis  toward  its  solu¬ 
tions,  and  provides  for  testing  and  evaluation  of  its  conclusions. 

At  this  conference  we  have  heard  discussed  many  of  the  scientific 
techniques  which  are  a  part  of  this  management  tool.  These  techniques  such 
as  linear  programming,  probability  statistics  and  value  theory  make  possible 
the  establishment  of  new  facts  and  relationships.  They  permit  investigation 
between  many  facets  of  a  problem  and  between  the  problem  and  other  related 
problems . 

Operations  research  is  thus  a  tool  which  affords  new  dimensions  to  aid 
the  logistician  in  the  solution  of  his  problems.  Accepting  this  fact,  what 
is  the  scope  of  the  Army’s  logistical  management  problems  which  this  opera¬ 
tions  research  tool  can  help  to  solve? 


198 


Fourth  Conference  on  Operations  Research 


Army  logistics  are  global  in  nature ,  They  encompass  the  control  of 
annual  expenditure  of  $U«5  million  for  support  and  equipment;  the  management 
of  physical  inventories  valued  at  approximately  &2l;  billion  and  the  control 
of  some  300  major  installations. 

In  addition  to  these  current  operational  responsibilities  and  associated 
problems,  the  Army  logistician  is  concerned  with  logistical  planning  problems 
for  future  combat  and  zone  of  interior  operations.  In  fact,  probably  the 
most  difficult,  complex  and  urgent  problem  facing  the  logistician  is  how 
to  establish  future  logistics  doctrine  and  combat  support  in  the  face  of 
rapidly  changing  tactical  doctrines,  explosive  political  environments  and 
economic  pressures. 

These  latter  factors  are  the  resultants  of  three  current  developments 
which  are  having  major  military  impact. 

First,  the  revolution  in  science  and  technology  is  ushering  in  many 
new  and  complex  weapons.  These  scientific  innovations  cause  major  changes 
in  tactical  and  strategic  doctrine.  Second,  industrially  backward  nations 
are  eager  to  secure  the  benefits  from  modem  civilization.  They  are  vulner¬ 
able  to  internal  political  disturbances.  They  generate  international  economic 
political  and  military  tensions,  and  thirdly,  we  are  undergoing  a  period  of 
rising  costs.  We  are  experiencing  all  of  the  frustrations  which  these  spirall¬ 
ing  costs  produce -relationships  to  national  economy,  international  economic 
competition,  social  reforms  and  political  readjustments. 

Against  this  panorama  of  major  developments,  the  art  and  science  of  war¬ 
fare  are  obviously  changing.  The  operational  concepts  of  tactics  and  strategy 
are  necessarily  fluid. 

For  the  logistician,  this  situation  of  fluid  tactics  and  new  developments  - 
scientific,  social,  political  and  economic  -  have  created  problems  of  a 
magnitude  not  previously  encountered. 

These  problems  are  too  complex  for  the  logistician  to  resolve  solely 
from  past  experience.  These  problems  involve  too  many  facts  and  too  many 
facets  which  have  not  counterparts  in  past  experience. 

Logistical  studies  have  been  made  in  the  Army  for  years,  but  today*  s 
studies  require  a  greater  heed  for  systematic  and  comprehensive  analysis 
of  broader  and  longer  range  problems. 

Necessarily,  increased  emphasis  is  being  placed  on  the  application  of 
scientific  techniques  as  an  aid  to  logistical  decision  making* 

The  application  of  these  scientific  techniques  in  terns  of  DA  logistics 
research  studies  are  provided  through  two  sources  -  contracts  with  civilian 
commercial  or  educational  organizations,  and  through  so  called  in-house 
studies  performed  by  military  organizations.  The  organizational  control  of 
this  study  effort  is  vested  in  the  Chief  of  Research  and  Development,  DA. 

The  latter  has  cognizance  of  all  Army  research  studies.  This  responsibility 
of  the  CR&D  includes  responsibility  for  insuring  coordination,  integration, 
avoiding  duplication  and  assuring  optimum  utilization  of  research  study 
results. 
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The  Army  Research  Office,  an  integral  part  of  OCR&D,  has  been  established 
as  the  focal  point  for  insuring  accomplishment  of  an  integrated  research 
program  responsive  to  the  growing  demands  of  the  Army. 

In  the  fields  of  non-materiel  logistics  research,  the  CR&D  relies  upon 
the  guidance  and  coordination  furnished  thru  the  ODCSLOG, DA.  It  is  through 
this  guidance,  coordination  and  collaborating  with  ODCSLOG  that  the  direction 
and  supervision  of  the  Army  non-materiel  logistics  research  efforts  are 
exercised. 

Much  of  the  detailed  operational  aspects  of  non-materiel  logistics 
research  -  programming,  administration,  and  supervision  -  are  performed 
by  the  Army  Logistics  Management  Center,  Ft.  Lee,  Virginia,  a  field  agency 
of  DC SLOG. 

The  Array  Logistics  Management  Center  conducts  such  studies  as  are  re¬ 
quired  to  assist  DCSLOG  in  developing  guidance  for  future  logistics  organiza¬ 
tions,  systems  and  procedures,  and  to  assist  the  CG,  USCONARC  in  the  formula¬ 
tion  of  logistics  organizations  and  doctrine  for  the  Theater  Army.  In  per¬ 
forming  these  tasks,  .the  ALMC  participates  in  the  development  of  the  logistics 
research  programs  of  the  Technical  Services.  In  addition,  ALMC  has  been 
designated  as -  the  office  of  record  for  all  information  pertaining  to  past 
DA  logistical  studies,  current  DA  studies  and  information  pertaining  to  the 
current  logistical  study  efforts  by  the  Navy,  AFf  other  government  agencies 
and  commercial  organizations.  I  suggest  that  each  of  you  having  responsibili¬ 
ties  in  the  logistics  research  area,  establish  the  necessary  liaison  with 
personnel  of  the  LR&D  Division  of  ALMC,  Ft.  Lee,  Va, 

In  terms  of  logistic  study  programming  responsibilities,  the  DCSLOG  is 
primarily  concerned  with  those  projects  having  Army  wide  logistical  interest. 
The  CG,  USCONARC  efforts  are  centered  on  field  Army  operational  needs;  and 
the  Chiefs  of  Technical  Service  studies  are  devoted  to  their  particular 
specialty  fields. 

The  basic  objective  of  these  Amy  logistics  research  programs  is  to 
assist  in  the  development  of  a  logistics  systems  which  will  be  responsive 
to  the  growing  strategic  and  tactical  operational  requirements  of  the  Amy. 
These  studies  necessarily  deal  with  the  nature  of  future  warfare.  Since 
future  war  is  a  broad  subject  and  study  efforts  are  both  expensive  and 
time  consuming,  objectivity  end  guidance  are  necessary. 


The  basic  DA  regulation  covering  logistics  research  is  Logistics  Direc¬ 
tive  No,  136-1,  ODCSLOG,  subject  "Logistics  Management  Type  and  Operations 
Research  Projects"  dated  27  February  19$9.  This  regulation  establishes  the 
policies  under  which  contractual  studies  will  be  initiated,  the  contract 
approval  authorities,  responsibilities  for  supervision  of  the  studies  and 
procedures  for  necessary  study,  reporting  and  evaluation. 
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In  terms  of  monetary  effort,  the  combined  FY  59  non-materiel  logistics 
research  programs  of  DCSLOG,  USONARC  and  the  Technical  Services  amounts  to 
approximately  $3,000,000.  Some  of  the  principal  contractors  include  0R0  of 
Johns  Hopkins  University;  Harbridge  House,  Inc;  George  Washington  U;  Melpar, 
Inc;  Stanford  Research  Institute,  Technical  Operations  Inc;  MIT,  Planning 
Research  Corp  and  others. 

The  principal  subject  matter  covered  in  these  studies  include  inventory 
management,  financial  management,  maintenance,  transportation,  gaining  and 
control  systems. 

I  "will  be  the  first  to  admit  that  our  current  program  does  not  meet 
the  standards  which  we  would  like.  It  was  precisely  for  this  reason  that 
ALMC,  LR&D  Division  was  established  in  order  that  proper  study  programming 
and  control  of  studies  could  be  accomplished.  At  present  we  do  not  feel 
that  the  desired  degree  of  integration  of  study  efforts  is  being  achieved. 
Adequate  inventories  of  past  studies  have  not  been  established;  and  proper 
evaluation  and  utilization  of  past  efforts  have  not  been  attained.  Some 
of  these  shortcomings  can  and  are  being  corrected  without  too  much  difficulty. 
Other  problem  areas  not  so  easily  corrected  include  the  following;  first, 
it  is  difficult  to  convince  operating  personnel  of  the  value  of  log  research. 

It  is  difficult  to  demonstrate  to  them  the  value  of  objective  evaluation. 
Considerable  effort  will  be  necessary  to  overcome  this  obstacle.  Secpndly, 
there  is  a  lack  of  communication  between  the  operating  personnel  (those  who 
have  problems)  and  the  research  personnel  attempting  to  assist  in  the  problem 
solution.  This  lack  of  mutual  understanding  appears  to  stem  from  a  failure 
to  define^  the  problem  adequately  in  turns  of  management  and  research  personnel 
appreciation,  and  the  improper  selling  of  the  merits  of  operations  research. 
Lastly,  there  is  the  failure  to  translate  the  study  findings  into  operational 
use.  This  is  a  corollary  of  the  failure  to  communicate  between  the  operational 
and  research  personnel.  Many  of  our  past  studies  were  published  and  placed 
in  some  file.  There  has  been  little  effort  to  translate  these  studies  into 
practical  applications.  Nothing  will  kill  a  research  program  so  fast  as  a 
failure  to  provide  a  payoff.  Consequently  our  policy  on  new  studies  is 
weighed  heavily  in  terms  of  practical  applications. 

My  remaining  comments  will  be  devoted  to  future  research  programs.  As 
previously  stated,  the  ALMC  is  the  operating  organization  primarily  concerned 
with  the  devlopment  of  Army-wide  type  logistical  study  programs.  From  a 
policy  viewpoint,  the  studies  will  be  oriented  toward  future  wars  and  emphasis 
will  be  placed  on  those  studies  which  offer  a  high  probability  of  practical 
applications.  The  tentative  FY  60  program  consists  of  the  following  areas: 

a.  Continued  development  of  logistics  gaming  capability.  This  capa¬ 
bility  is  necessary  as  a  training  device,  as  a  means  for  testing  logistic 
plans,  and  as  a  means  for  evaluating  alternative  systems  for  maintenance 
and  supply. 

b.  Continued  efforts  in  logistics  economics.  These  studies  include 
the  fundamental  analysis  needed  to  provide  policy  guidance  in  methods, 
procedures  and  trends  in  the  basic  functional  areas  of  logistics,  i.e.,  in¬ 
ventory  moderization,  economic  inventory  policy,  economics  of  equipment 
maintenance  and  overhaul,  plant  and  production  equipment  obsolescence, 
provisioning  policy,  and  effects  of  financial  controls. 
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c.  Organization  and  systems  analysis.  These  studies  include  elements 
of  supply  responsiveness,  i.e.  optimum  CONUS  logistics  complex.  Theater  Army 
supply  and  maintenance  organizations,  applications  of  ADPS,  logistics  operations 
in  support  of  limited  war,  missile  supply  and  support  systems,  and  strategic 
lift. 


d.  Development  of  realistic  operational  and  administrative  logistical 
data.  These  studies  consist  of  revision  to  EM  101-10  "Staff  Officers 

Field  Manual,"  development  of  maintenance  factors,  and  development  of  missile 
supply  rates. 

e.  Lastly,  our  program  includes  the  analysis  of  the  impact  of  techno¬ 
logical  developments  upon  logistical  systems,  i.e.  impact  of  missile  supply 
and  support  upon  logistic  concepts,  and  the  effects  of  non-destructive  uses 
of  atomic  energy  upon  logistic  systems. 

In  summary,  the  purposes  of  my  discussion  with  you  have  been  l)  to 
impress  upon  you  the  importance  of  logistics  research  and  2)  to  provide 
insight  into  the  DA  logistical  research  organizations,  policies,  programs 
and  problem  areas.  I  am  convinced  of  the  importance  of  logistics  research 
and  the  need  to  devote  greater  portions  of  our  resources  to  the  program 
objective  areas  which  I  have  outlined..  I  hope  that  those  of  you  concerned 
with  sponsoring  research  or  conducting  research,  will  give  more  thought  and 
attention  to  the  problems  of  communication  between  the  operators  and  re¬ 
searchers,  and  to  the  translation  of  study  results  into  practical  applications. 


FUTURE  OPERATIONS  RESEARCH  PROBLEMS  IN  ORDNANCE  LOGISTICS 


Herbert  P.  Galliher 
Massachusetts  Institute  of  Technology 

When  I  was  asked  to  talk  originally,  it  was  on  the  subject  of  future 
Operations  Research  problems  in  Ordnance,  and  that  was  too  much  for  me  so 
I  suggested  the  word  logistics  be  added,  but  Colonel  Leist  has  now  described 
logistics  in  such  a  way  that  the  subject  is  still  much  too  large  for  me0 
In  fact  it  seemed  to  me  that  the  way  he  has  described  it  is  that  it  is  not 
even  a  matter  for  Ordnance  at  all,  or  even  for  the  Army,  and  this  comes  out 
not  really  as  an  attempted  bit  of  humor  but  merely  the  statement  of  the  facts, 
I  also  feel  a  little  bit  like  a  person  might  feel  if  the  situation  were  as 
follows:  A  group  of  people  undertook  to  cut  down  a  mountain  with,  we'll  say 
shovels.  After  a  few  initial  spadefulls  an  invited  speaker  was  brought  in 
to  discuss  the  subject  of  the  future  work  in  cutting  down  the  mountain. 
However,  just  before  him  there  appeared  on  the  program  a  representative  of 
the  boss  who  said  what  the  future  work  was  going  to  be.  In  a  similar  fashion 
it  seems  to  me  that  Colonel  Leist  gave  a  good  coverage  of  future  work  in 
this  area. 

Now,  I  am  not  a  member  of  the  Ordnance  corps,  although  we  have  in  our 
work  at  MIT  been  very  generously  supported  by  the  Ordnance,  but  in  reference 
to  what  I  have  to  say  now  I  would  like  to  feel  that  I  am  completely  indepen¬ 
dent,  that  I  can  say  what  I  want  to  say  with  impunity.  If  I  feel  things  are 
not  being  done  right,  I  am  going  to  say  so,  I  am  going  to  try  to,  and  see 
what  we  can  turn  up  of  value  to  us  in  our  thinking.  I  shall  think  in  this 
way,  and  try  to  get  you  to  share  this  attitude  with  me  for  a  little  while 
this  afternoon.  It  seems  to  me,  incidentally,  that  it  would  be  appropriate 
in  the  last  session  of  this  conference  for  me  not  to  give  a  talk,  but  as 
soon  as  possible  for  us  to  really  address  ourselves,  perhaps  as  a  group,  to 
an  open  discussion  of  this  question  which  is  the  subject  of  the  hour.  In 
order  to  try  and  accomplish  that,  I  am  going  to  say  some  things  which  I 
hope  will  provoke  discussion. 

Now,  let  me  take  a  narrow  aspect  of  logistics  in  Ordnance  first  off, 
and  think  of  the  matter  of  supply.  Actually  since  the  bulk  of  our  work  for 
Ordnance  has  been  done  in  connection  with  secondary  items,  and  the  study 
of  the  supply  of  these,  particularly  at  the  national  level,  you  are  being 
warned  that  there  may  be  a  bias  in  my  attempting  to  talk  about  supply  in 
any  general  terms.  But  if  we  look  at  the  Ordnance  supply  system  as  a  whole, 
it  seems  to  me  that  it  has  one  function,  namely  to  link  the  manufacturer  or 
the  designer  -  the  people  who  can  make  things  -  with  the  demand.  The  latter 
includes  the  people  who  need  them  and  the  kind  of  organization  for  combat 
or  for  threatened  combat  in  peace  -  time  that  we  need.  But  the  job  of  Ord¬ 
nance  in  its  supply  function  is  to  correlate  the  agency  that  supplies  and 
the  agency  that  uses. 

The  entire  design  of  Ordnance  both  as  to  people,  positions,  jobs,  classi¬ 
fication,  the  amount  of  effort  as  to  depots,  where  they  should  be,  whether 
they  should  be,  as  to  what  we  should  stock  there,  whether  we  should  stock  or 
not,  as  to  posts,  camps,  and  stations,  whether  there  should  be  stocking  points, 
or  not,  etc.,  all  are  part  of  the  picture.  Of  all  the  basic  questions  of 
supply  (and  there  are  usually  about  four  dominant  questions  -  what  items  to 
stock,  where  to  stock  therr^ when  to  stock  them, and  how  much).  The  answer  to 
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all  these  questions  is  an  answer  to  the  question  of  how  Ordnance  should  ope¬ 
rate®  We  have  only  to  measure  the  operation  in  terms  of  its  performing  the 
fundamental  function,  namely,  linking  the  supplier  with  the  user®  In  the  end, 
all  our  present  establishments  might  go  by  the  board  if  we  found  that  an 
alternative  organization  of  the  system  better  performed  this  function  of 
linking  the  supplier  to  the  user* 

This  problem  is  just  obvious  every  time  you  turn  around,  the  difficulty 
of  doing  this  and  the  need  of  doing  it.  We  even  have  the  problem  in  linking 
supplier  and  user  right  in  our  research  work.  We  cannot  keep  track  of  the 
research  projects  that  are  going  on,  and  who  is  doing  what  on  what  project, 
or  what  has  been  done,  or  what  the  findings  were*  We  cannot  even  coordinate 
this  information  in  our  own  research  work*  It  is  easy  to  see  that  a  problem 
of  the  size  of  supply  throughout  Ordnance  which  in  spare  parts  alone,  I 
guess,  is  nearly  half  a  billion  dollars  worth  of  issues  a  year  is  really  a 
major  problem,  and  therefore  must  be  a  big  problem  of  coordination®  As  a 
matter  of  fact  to  the  extent  that  we  failed  to  perform  the  right  linkage, 
we  are  failing  to  perform  the  function  as  well  as  we  might.  One  of  the  main 
aspects  is  communication,  how  can  we  communicate  faster.  Thus  we  cannot 
solve  the  logistics  problem  until  we  can  communicate  faster  and  better,  un¬ 
til  we  can  store  infoimation,  more  of  it  in  retrievable  form  -  quickly  re¬ 
trievable. 

These  general  considerations  are  not  merely  general  but  come  up  specifi¬ 
cally  in  any  area  in  which  you  attempt  to  work.  Let  me  illustrate  this  with 
respect  to  the  matter  of  spare  parts  supply  on  which  we  have  done  some  work, 
not  only  us  but,  of  course,  a  lot  of  other  people.  If  you  look  at  the  supply- 

system  as  a  whole,  you  have  a  map  of  mainly  the  United  States,  at  least  as 

far  as  source  of  supply  goes,  and  perhaps  I  can  do  this  by  drawing  a  sketch 

on  the  board.  If  this  is  the  map  of  the  United  States,  and  if  we  have  our 

manufacturer  or  potential  manufacturer  located  here,  there,  and  everywhere, 
we  have  also  our  users  and  they  are  located  here,  there,  and  everywhere,  not 
only  on  this  map  but  on  the  map  of  the  world.  Now  the  national  supply  pic¬ 
ture  and  the  structure  of  it  so  far  is  that  we  divide  out  items  into  commodity 
groups.  We  divide  them  into  six  -  seven  now,  I  guess.  We  have  also  ware¬ 
houses  or  major  depots,  and  thereby  we  have  really  divided  the  map  into  a 
number  of  depot  areas.  In  each  area  we  have  at  least  a  major  depot  and  cus¬ 
tomers,  who  are  pretty  well  scattered.  Items  are  routed,  however,  through 
posts,  camps,  and  stations,  or  direct  support  units  of  which  there  are  several 
hundred,  and  these  in  turn  are  assigned  to  a  depot  for  normal  supply  of  items. 
The  depot  here  supplies  demand  if  it  has  the  stock.  If  it  doesn't,  the  demand 
is  referred  to  the  system  of  all  the  depots,  and  may  be  supplied  from  one  of 
the  other  depots,  or  it  may  be  that  the  demand  waits  until  stock  on  the  way 
from  the  manufacturer  -  a  particular  manufacturer  for  this  stock  -  is  being 
allocated  to  all  the  depots. 

That  is  the  geographical  picture.  Now  the  picture  in  time  is  basically 
one  of  intermittent  replinishment  of  the  system.  That  is  to  say,  every  so 
often  we  have  a  batch  of  stock  manufactured  by  the  manufacturer*  Once  in  a 
while  we  give  him  an  order,  be  it  every  six  months  or  every  year  or  with 
some  other  variationo  In  a  few  cases  we  give  standby  orders  to  replinish 
this  system  on  a  continuing  basis,  but  essentially  the  time  picture  is  one 
of  movements  at  some  point  like  this,  and  on  occasion  a  spread  like  that. 
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Meantime,  lots  of  requests  within  the  system  requiring  transfers  of  stock 
back  and  forth  between  units  come  along.  If  you  try  to  draw  a  very  simple 
picture  of  the  operation  as  a  whole,  this  is  the  sort  of  picture  that  you 
get. 

Why  should  we  draw  a  simple  picture?  Because  this  is  one  very  important 
problem  area  that  we  ought  to  keep  working  on.  The  most  important  area,  it 
seems  to  me,  is  the  system  area,  the  large  area,  the  operation  of  the  entire 
business.  There  are  obvious  reasons  why  that  is  true.  In  the  first  place 
the  business  is  pretty  big.  It  is  easy  for  it  to  get  sluggish.  It's  a  diffi¬ 
cult  thing  to  work  on  and,  therfore,  any  work  on  it  is  apt  to  pay  off  in  a 
big  way.  Second,  the  risk  of  not  working  on  it  is  very  great,  but  in  order 
to  work  on  a  big  system  we  have  to  have  a  simple  model  for  it,  and  so  we  have 
to  try  to  get  a  picture  of  it.  Once  we  get  a  picture  like  this  we  can  ask 
questions,  obviously,  lots  of  questions. 

Let’s  list  some.  How  should  the  manufacturer  be  selected?  When  should 
he  be  selected?  Well,  we  have  lots  of  Army  regulations  and  lots  of  pressure 
from  Congress  which  dictate  the  way  in  which  manufacturers  are  selected. 

There  are  all  sorts  of  bases  on  which  selections  are  made.  Cost  is  an  element 
integrity  is  another;  political  pressure  is  another;  national  economy  is 
another.  All  of  these  criteria  dictate  selection.  How  is  each  selection 
made?  Well,  there  are  various  review  boards  at  which  these  considerations 
are  taken  into  account.  How  do  the  manufacturers  know  that  a  selection  is 
being  made?  Well, there  is  a  procedure  for  advertising  in  some  cases,  for 
not  advertising  in  others.  I  dare  say  a  good  many  manufacturers  could  make 
things  if  they  knew  that  there  was  an  opportunity  to  make  them.  Now,  then 
as  we  come  into  the  system  itself,we  see  this  system  of  depots,  and  questions 
arise.  How  many  depots  should  we  have?  Where  should  we  stock  these  items? 
What  items  should  we  stock?  This  is  a  terribly  messy  question.  Nobody  seems 
to  have  resolved  this  at  all.  We  have  all  kinds  of  stockage  lists.  A  most 
certain  thing  about  a  stockage  list  is  that  any  item  that  is  on  it  now  has 
a  chance  of  not  being  on  it  three  months  from  now  or  six  months  from  now. 

They  are  very  unstable  lists.  When  should  we  stock?  How  should  we  move 
our  stuff  around?  How  much  should  we  stock  at  posts,  camps,  and  stations? 
These  are  all  really  basic  questions  to  which  we  have  very  few  answers  so 
far. 

But  perhaps  we  have  a  few.  As  a  result  of  some  work  we  have  done,  we 
have  got  a  picture  for  ourselves  of  the  operation  of  the  depots.  We've  got 
a  mathematical  picture  of  it  and  we  have  used  it  to  characterize  the  depots 
under  these  frequent  replinishment  schemes.  We  know  what  will  happen. 

We've  got,  as  a  matter  of  fact,  a  simulation  of  the  depot  system  that  we 
constructed  on  the  70l*  computer,  which  will  take  any  given  set  of  secondary 
items,  and  generate  by  random  numbers  each  typical  demand  that  will  occur 
in  a  year  at  any  depot,  will  handle  it  according  to  certain  rules,  will 
cause  inter-depot  shipments,  will  cause  items  to  be  manufactured,  etc.  In¬ 
cidentally,  the  simulation  will  tabulate  a  great  deal  that  happens,  how  much 
waiting  was  incurred  by  the  people  who  ask  for  items,  how  often  the  demand 
was  met  at  the  initial  source,  how  often  at  some  other  source,  how  much  was 
shipped  back  and  forth  between  manufacturers.  The  simulation  will  do  all 
this  and  give  us  a  year's  experience  in  one  second  on  the  70U»  Well,  from 
this  we  got  a  picture  of  just  what  will  happen  under  this  kind  of  operation 
which  satisfies  us  for  the  moment.  Consequently, I  would  like  to  look  away. 
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that  is j  either  down  towards  posts,  camps,  and  stations,  or  up  towards  manu¬ 
facturers.  Perhaps  our  next  model  should  be  a  model  of  the  manufacturers 
and  of  how  many  manufacturers  are  needed,  and  how  they  should  be  located. 

If  so,  we  must  put  in  the  model  the  constraints  the  politicians  would  like 
to  have  satisfied,  because  if  we  can’t,  we’ll  never  sell  it  to  the  politicians. 
Thus  if  we  look  up,  we  observe  the  problem  of  locating  manufacturers . 

By  the  way,  if  we  propose  to  do  a  model  of  this,  we  have  in  mind  ultimately 
a  data  processing  system  which  will  have  stored  in  it,  and  using  in  its 
operation,  everything  that  we  use  in  the  model  of  the  manufacturer.  Whatever 
its  capacity  is,  certainly  it  won’t  be  easy  to  describe  the  capacity  of  a 
manufacturer  in  terns  of  a  small  number  of  numbers,  so  we  shall  have  to  do 
some  extremely  clever  work  in  order  to  characterize  the  capacity  of  manu¬ 
facturers  against  a  potential  stock  list  of  somewhere  between  25,000  and 
150,000  or  more  items  depending  on  the  frequency  with  which  they  are  demanded. 

At  any  rate,  we’ll  have  some  job  of  doing  that.  But  ultimately  this  is  the 
kind  of  linkage  that  we  would  like  to  take  advantage  of,  just  as  I  would 
like  to  be  able  to  know  right  now  who  is  doing  what  work  on  this  problem 
anywhere  in  the  United  States.  If  I  could  have  the  answer  to  that  question 
in  a  hurry,  I  might  do  my  work  entirely  differently.  We  might  save  a  year’s 
time  in  what  we  did,  but  I  can't  get  the  answer  to  that  question  very  easily, 
and  so  I  might  just  as  well  go  ahead  and  solve  the  problem  for  myself.  A 
lot  of  the  time  that’s  what  happens  when  you  buy  something  from  the  manu¬ 
facturer.  Xou  say,  sure,  if  we  advertise  this  thing  widely  enough  we  might 
pick  up  another  10  per  cent  in  cost,  but  we  just  don't  do  it.  It  costs  us 
too  much  to  advertise.  We  might  reduce  costs,  also,  if  we  had  a  very  good 
formula,  objective  formula.  This  might  cut  out  lots  of  the  complaints  from 
our  politicians.  Well,  here  is  an  area  into  which  we  ought  to  work,  it  seems 
to  me. 

Now  we  might  go  down  the  other  way  towards  the  posts,  camps,  and  stations. 
Mr.  Pear,  who  is  here  from  the  Signal  Corp,  was  saying  earlier  in  the  day  'to 
me  that  this  is  an  important  area.  One  of  the  big  questions  in  this  area  is 
how  posts,  camps,  and  stations  should  be  supplied.  Should  they  have  a  port 
which  is  resupplied  or  should  it  be  a  stocking  point?  Where  should  we  re- 
supply  from?  Let's  not  talk  just  about  supply  on  one  hand  and  maintenance 
on  the  other  because  these  are  both  part  of  supply  function.  We  don’t  care 
how  we  get  this  part,  or  how  we  get  it  into  working  condition,  whether  it 
is  by  supply  or  manufacturer  or  by  maintenance.  So  here  then  we  may  ask 
ourselves  what  are  other  alternatives  of  operation  which  might  be  more  prof 3  t- 
able  and  which  would  be  feasible.  Feasibility,  incidentally,  rests  a  lot 
nowadays  on  the  use  of  computers,  and  it  ought  to  because  our  ability  in  this 
area  is  increasing  very  rapidly. 

I  would  like  to  sketch  very  briefly  to  you  a  study  that  we  are  main  ng 
now  on  the  use  of  computers  in  one  area  of  this  operation.  This  will  be 
a  controversial  subject,  I  know,  so  we  will  have  some  fun  there.  As  a  result 
of  our  inventory  studies,  we  found  in  our  simulations  that  what  people 
were  saying  was  true  about  the  delay  between  the  time  that  a  customer  at  a 
post,  camp  or  station  asks  for  an  item  and  the  time  that  the  item  was  shipped 
from  some  depot  that  had  it.  Quite  often  it  would  be  the  same  level  depot. 
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but  about  thirty  per  cent  of  the  time  it's  not,  which  is  a  high  fraction. 

That  delay  is  pretty  substantial.  In  fact  if  the  item  is  not  shipped  from 
this  depot,  the  delay  is  between  thirty  and  forty-five  days,  between  the 
time  he  asks  for  it  and  the  time  the  shipment  was  made. 

Now  if  you  go  looking  into  this  delay,  what  you  go  looking  into  is  the 
entire  data  processing  operation  from  the  time  that  he  put  in  his  demand. 

Well,  at  present  it  is  a  manual  operation.  Cl  suppose  a  good  many  of  you 
are  much  more  familiar  with  it  than  I  am,)  in  which  he  periodically  sends 
in  a  list  of  items  that  he  wants.  This  is  called  a  hard-copy  requisition, 
and  then  this  is  broken  down  into  individual  requests  for  individual  items. 
These  are  put  through  files  at  the  depot,  sorting,  editing,  seeing  if  his 
whole  order  can  be  made  up,  etc.  If  his  whole  order  can’t  be  made  up,  de¬ 
lays  occur,  In  particular  it  generally  takes  about  three  days,  or  it  did 
take  about  three  days  for  that  requisition  to  go  through  that  operation  right 
there,  even  if  the  stock  was  there 0 

Now  if  the  stock  was  not  there,  this  is  what  would  happen.  And  notice, 
as  I  say,  something  like  thirty  per  cent  of  the  time  the  demand  was  being 
filled  from  somewhere  else,  (As  a  matter  of  fact  that  might  be  a  good  way 
of  operating  in  itself  if  you  visualize  a  combat  operation,  it  might  very 
well  be  true  that  more  than  thirty  per  cent  of  the  time  the  demand  was  not 
filled  from  the  primary  source  to  which  this  unit  was  assigned.  So  you  simply 
cannot  say.  ’that  percentage  is  too  high’.  It  might  profitably  be  higher 
than  that.)  At  any  rate,  what  happens  now,  in  the  thirty  per  cent  of  the 
time,  is  that  there  are  several  modes  of  operation.  If  this  is  a  request 
from  overseas,  from  the  European  Theatre,  say,  it  would  get  pretty  fast  handl¬ 
ing,  maybe  as  fast  as  two  days.  If  it  is  not  from  overseas,  in  fact  if  it 
is  a  routine  Anriy  continental  request,  it  will  probably  get  about  a  twenty- 
five  day  delay,  in  which  a  lot  of  things  have  gone  on  in  the  meantime.  The 
depot  doesn’t  have  the  stock  so  he  goes  to  the  Supply  Control  Point  and  says 
we  don’t  have  the  stock  -  we  have  the  demand.  The  Supply  Control  Point  looks 
at  the  stocks  at  the  other  depots.  They  say  so  and  so  has  the  stock,  try 
him.  So  he  tides  so  and  so.  But  in  the  meantime,  some  days  have  gone  by, 
and  by  the  time  our  man  tries,  so  and  so  no  longer  has  the  stock, 

fou  see,  this  typically  happens  when  everybody  is  short  of  stock  toward 
the  end  of  some  procurement  cycle.  Thus  you  have  the  problem  that  your  in¬ 
formation  is  never  up-to-date,  and  the  action  taken  on  it  is  action  taken 
on  something  out  of  date,  (and  sometimes  these  delays  don't  ever  converge!) 
Well,  these  delays  are  pretty  considerable. 

Now,  consider  one  other  little  matter.  Suppose  this  were  war  time,  and 
suppose  we  were  operating  under  such  an  organization  in  a  Field  Army  Theatre. 
What  kind  of  delays  could  we  tolerate?  I've  made  my  own  calculations  on.’ this 
as  a  result  of  participating  in  some  studies  involving  this  kind  of  operation. 
It  turns  out  that  the  data  processing  delay,  the  tolerable  delay,  in  such 
operations,  is  extremely  short.  Now,  if  it  is  short  and  if  moreover  we  are 
having  substantial  delays  now,  what  can  we  do  about  it? 
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In  our  work  at  MIT  we  are  studying  several  schemes  of  computer  networks 
to  do  this .  Let  me  sketch  one  just  very  briefly,  which  is  not  necessarily 
one  that  I  am  recommending.  But  it  will  give  us  a  feeling  for  the  problem. 
According  to  some  of  our  estimates  if  you  had  a  709  computer  located  somewhere- 
it  might  not  always  be  the  same  709  from  hour  to  hour  -  (we  could,  if  we  had 
a  telephone  type  network  linking  every  post,  camp,  and  station  directly  to  that 
unit,  and  linking  this  unit  to  every  depot),  then  roughly  the  following  would 
happen.  This  computer  would  have  to  handle  about  two  demands  a  second  under 
a  pretty  heavy  load  on  a  sixteen  hour  a  day  basis,  five  days  a  week.  It 
would  have  a  capacity  of  upwards  of  ten  demands  per  second  during  that  same 
period  of  time.  The  utilization  of  the  telephone  lines  would  amount  to 
somewhere  between  two,  say,  and  ten  telephone  lines  full  time  utilization. 

(We  could  not  make  full  utilization  of  the  telephone  lines.)  One  thing: 
there  would  not  be  a  punch  card  in  the  system.  We  are  sure  that  if  we  put 
punch  cards  in  the  system,  somebody  would  get  their  hands  on  them  and  in¬ 
terrupt  the  speed.  So  this  will  be  completely  electrical.  The  requests  put 
in  here  will  be  put  in  electrically  on  something  better  than  the  transreceiver. 
This  type  of  operation  would  be  on  to  tape  right  now. 

This  conputer  point  would  also  be  connected,  by  the  way,  to  the  Supply 
Control  Points .  In  fact  there  are  about  ten  basic  types  of  information 
that  would  pass  over  such  a  network,  and  a  demand  is  only  one  of  them.  The 
others  are  induced  massages.  You  have  instructions  to  ship;  you  have  reports 
of  shipment.  You  have  warehouse  refusals.  You  have  requests  for  content  of 
file.  You  have  an  instruction  from  the  supply  Control  Point  to  change  the 
parameters.  What  is- this  conputer  doing,  by  the  way?  Well,  it  could  be 
managing  these  items  according  to  the  rules  used  in  our  simulation.  They 
are  not  perfect  rules,  but  they  don't  seem  to  be  too  bad,  at  least  to  handle 
a  good  part  of  the  business.  What  kind  of  response  could  we  get  on  this? 

This  is  very  interesting.  If  we  had  to  handle  150,000  items,  that  the  stock 
fund  lists,  it  turns  out  that  it  would  take  us  a  number  of  hours  because 
each  time  a  demand  comes  in  we  have  to  pick  out  the  files  on  this  item  un¬ 
less  we  batched  items.  If  we  batched  items,  we  might  cut  that  time  down. 

But  let's  say  that  it  might  be  four  hours,  just  as  a  figure,  between  the 
time  that  the  demand  was  entered  here  on  the  system  and  the  time  that  a  ship¬ 
ping  instruction  was  printed  out  at  a  depot,  that,  according  to  the  file  here, 
had  the  stock.  However,  suppose  we  only  had  to  handle  25,000  items.  Why  I 
picked  25,000  is  that  25,000  represents  a  good  portion  of  the  active  items 
that  generate  most  of  the  demands.  Then  we  can  respond  in  a  matter  of  sec¬ 
onds.  Very  interesting. 

Wow,  why  do  I  suggest  this  type  of  unit?  Wot  because  we  are  recommend¬ 
ing  any  such  unit.  Why  do  I  suggest  this  type  of  network?  We're  not  recom¬ 
mending  it.  We're  going  on  and  study  some  others.  What  are  other  possibili¬ 
ties?  Other  possibilities  are  to  add  less  conputer  capacity  than  this,  but 
scatter  it  at  the  depots  and  at  other  points,  and  to  have  the  routing  of 
traffic  be  different.  What  do  we  have  to  explore  here?  What  we  have  to  ex¬ 
plore  are  the  alternatives  of  operation,  and  for  each  alternative  we  have 
to  conpute  the  operating  requirements  or  specifications.  That  is,  what  the 
network  would  have  to  do,  how  much  traffic  of  each  sort  it  would  have  to  han¬ 
dle  and  at  what  rates,  what  kind  of  congestion  we  could  expect  from  over¬ 
loads.  Conpute  these  requirements  for  each  one  of  these  alternative  network 
designs,  and  simply  compare  their  performance  in  terms  of  response.  When 
we've  done  that,  what  we've  got  is  something  like  a  cost  versus  performance 
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curve  for  various  types  of  network.  From  hereon,  in  my  opinion,  we  can 
leave  it  up  to  the  Commanding  Officer  of  this  outfit  to  make  the  decision. 

Well,  here  is  an  area,  in  which  we  are  working  ourselves.  What  I'm  say¬ 
ing  is  that  here  is  a  system  that  is  operating  as  a  whole  in  which  there  is 
something  that  obviously  needs  to  be  done.  Speed  it  up.  The  quicker  we  do 
it,  the  better.  Actually  the  major  part  of  this  study,  it  seems  to  me,  is 
not  even  an  Operations  Research  study.  However,  we  are  going  to  compute  the 
congestion  factors  because  there  are  serious  problems  that  will  occur  here, 
such  as  queuing  problems.  I  think  the  main  job  to  be  done  here  is  to  get 
some  such  system  installed  just  about  as  quickly  as  possible.  Of  course, 
the  fact  that  we  can't  utilize  all  those  telephone  lines  in  this  operation 
means  somebody  else  can  utilize  them  -  somebody  in  the  other  tech  services. 

We  have  other  kinds  of  messages  coming  over  this  network.  Now  the  Signal 
Corps  has  the  job  of  communication  in  the  Army,  but  we  have  our  requirements, 
too  -  and  the  other  tech  services  have  their  requirements.  Our  problem  here 
is  to  get  something  done  quickly.  If  this  isn't  done.  Why  the  Operation  Re¬ 
search  work  that  was  done  will  be  kind  of  a  failure.  And  there  will  be 
another  weak  report  to  Congress  on  what  has  been  accomplished  by  Operation 
Research  in  Ordnance. 

Now  I  would  like  to  turn  briefly  to  another  problem  about  future  work 
in  Operations  Research,  and  this  haS  to  do  with  the  problem  of  measurement. 
Operations  Research  began,  I  think,  or  so  people  claim,  to  help  administrators 
make  decisions  according  to  some  objective  rules.  We  can  draw  you  a  qualita¬ 
tive  model  of  an  operation.  We  can  find  the  optimum  solution,  and  the  net 
result  of  what  was  done  is  really  this.  First,  they  restated  the  problem, 
and  redescribed  the  operating  problem  in  quantitative  terms.  Then  the  opti¬ 
mization  part  says  that  if  the  value  of  the  cost  is  this,  and  if  the  value 
of  the  demand  is  this,  and  if  the  value  of  the  need  is  this,  then  this  is 
the  optimal  strategy. 

Now  there  are  two  different  things  here  that  have  been  done.  The  first 
part  was  to  state  the  problem  in  quantitative  form.  The  second  is  to  try 
to  apply  it  so  as  to  optimize  something  in  any  given  situation.  If  that 
was  all  that  the  Operation  Researcher  should  have  to  do,  his  job  is  done. 

If  it  so  happens  that  nobody  can  measure  the  values  of  all  those  things, 
that  is  not  to  be  charged  against  him. 

Well,  I  don't  take  that  viewpoint.  Frankly,  if  somebody  can  show  that 
I  have  given  a  solution  to  this  form  and  he  can't  measure  the  value  of  X, 
the  value  of  Y,  the  value  of  Z,  he's  really  saying  well,  I've  been  talking 
to  myself  all  the  time.  I've  got  a  completely  hypothetical  theoretical 
problem  (and  this  is  very  basic  research  because  it  has  no  potential  appli¬ 
cation  at  all  and  that  fits  the  definition  code!)  If  it's  applied  research 
then  it  must  have  been  that  I  thought  he  could  measure  it.  And  in  fact, 

I  think  our  biggest  problem,  the  big  problem  nowadays  is,  how  to  find  the 
cost  of  procurement,  and  the  cost  of  inventory,  the  value  of  being  out  of 
stock.  What  are  these  values?  How  do  we  get  them?  If  we  don't  measure 
them,  then  the  application  is  not  going  to  be  made.  The  research  is  not 
going  to  be  implemented,  and  it  will  die,  says  Colonel  Leist,  and  I  am  sure 
he  is  right.  All  right,  I  think  we  must  take  on  the  challenge  of  measurement. 
For  example,  we  must  take  on  the  challenge  of  forecasting  demand,  because  our 
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models  for  inventory  control  require  that  we  forecast  the  demand.  Now  de¬ 
mand  forecasting  is,  afterall^  a  tough  problem.  Most  Operations  Research 
people,  the  mathematicians  in  particular,  will  not  take  on  this  job-  They 
are  'hypothetical'  people.  They  only  grind  out  solutions  of  the  formula 
'if  A,  then  B’  .  It  is  the  engineers  who  supply  the  value  of  A- 

Our  Operations  Research  approach,  it  seems  to  me,  has  got  at  this  point 
to  rescue  itself,  to  take  on  an  engineering  approach  and  measure  these  things. 
What  are  the  measures  and  how  can  we  measure  them  easily?  If  we  want  to  com¬ 
pute  the  cost  of  carrying  something  in  inventory  in  various  depots,  clearly 
we  can't  take  every  item  on  the  list  and  go  out  and  cost  all:  those  things. 

How  do  we  cost  them?  Oh,  we  watch  them  for  a  year  and  we  report  the  cost. 

We  index  these  against  other  things,  and  then  we  say  use  that  as  the  cost, 
use  the  index.  Well,  we  spent  an  awful  lot  of  work  on  it  and  besides  that 
we've  now  still  got  to  predict  the  index.  So  we're  back  to  the  same  old 
problem  again. 

I  think,  in  other  words,  it's  somewhat  less  than  what  you've  been  asked 
to  do  if  you  turn  out  a  theoretical  solution.  (Unless  your  job  is  perhaps 
what  ours  is  at  MIT,  which  is  to  turn  out  theoretical  solutions'.)  So  there 
are  really  levels  of  Operations  Research.  One  level,  and  let  me  just  dwell 
on  that  point  a  moment,  one  level  is  modeling  problems  as  a  whole .  I  would 
like,  at  this  point,  make  the  point  that  the  best  Operations  Research  is 
going  to  be  done  in  Ordnance  by  Ordnance  people,  by  you  people  who  are  already 
where  you  are.  It  is  not  going  to  be  done  by  people  who  come  in  to  do  it 
for  you.  It  is  not  going  to  be  done  by  your  Commanding  Officer  nor  your  su¬ 
periors  ;  it  is  going  to  be  done  by  you.  What  problems  will  you  solve?  All 
of  them.  The  ones  about  the  system  as  a  whole  -  yes.  Is  that  your  respon¬ 
sibility?  It  doesn't  make  any  difference;  if  you  want  to  see  a  job  get  done, 
do  a  job  on  it.  Present  it  at  the  next  Operation  Research  conference.  (Do 
it  nights'. ) 

Wow,  as  far  as  future  problems  go,  there  are  obviously  lots  of  applica¬ 
tions  that  are  going  to  be  made  that  I  couldn't  begin  to  predict  what  is  going 
to  happen.  We've  had  a  good  list  of  them  today  and  yesterday.  One  that  I've 
wondered  about  is  the  scheduling  of  ammunition  production  done  with  the  help 
of  some  mathematical  formula.  This  is  nearly  a  (linear)  transportation  type 
of  problem.  The  last  answer  I  had  was  that,  that  method  has  not  yet  been 
applied  to  this  problem.  I  suspect  the  people  who  make  the  decision  as  to 
the  schedule  are  in  the  Pentagon.  If  you're  not  in  the  Pentagon,  then  some¬ 
body's  problem  is  how  to  get  your  solution  to  the  attention  of  the  people  in 
the  Pentagon,  because  you  are  their  talent.  Linear  programming  applied  to 
a foundary  cost  operation  was  mentioned  this  morning.  There  must  be  lots  of 
others  like  that.  Incidentally,  the  question  ms  raised  then  about  the  use 
of  this  by  ether  foundries.  I  am  sure  that  a  lot  of  you  know  that  linear 
programming  has  been  used  by  the  oil  companies  in  a  very  similar  type  of 
problem,  namely  putting  in  mixtures  of  crude  and  getting  out  product  streams. 
This  is  a  matter  of  routine  in  a  number  of  oil  refineries.  There  is  another 
nice  area,  demand  forecasting*  Still  another  nice  area  is  product  characteri¬ 
zation.  That's  a  tricky  one  and  I  believe  no  models  exist.  I  would  like  to 
see  people,  who  know  products  and  who  also  know  some  Operation  Research,  work 
on  it .  For  example,  how  do  you  characterize  a  product  for  purposes  of  storing 
its  file  in  a  data  processing  system?  Well,  you  put  a  stock  control  number  in. 
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That's  an  identification  but  not  a  connotation.  Can  you  gain  anything  from 
the  connotation?  Can  we  build  a  product  characterization  which  will  at  least 
do  things  like  accounting  for  multiple  applications  for  substitute  ability? 
There  are  lots  of  points  at  which  our  descriptions  of  things  should  be  a  lot 
more  indeixical  than  symbolic.  There  is  a  great  tendency  in  government  to 
use  symbolic  descriptions  of  things  so  there  is  no  similarity  between  the 
name  of  a  thing  and  the  nature  of  a  thing. 

Well,  I've  made  just  a  few  suggestions  here.  Perhaps  one  other  thing 
ought  to  be  mentioned  and  that  is  the  value  of  information.  It  seems  to 
me  that  information  is  the  biggest  thing  that  we  need  and  that  it  has  a 
greater  value  than  we  think.  Let  me  give  you  an  example.  Suppose  that  a 
man  A  makes  something  for  a  man  B.  Now  the  reason  he  makes  it  is  so  B  can 
use  it  to  make  a  project.  Now  let's  suppose  A  invests  X  units  of  effort, 
and  that  if  B  knows  that  he's  got  it,  he  can  go  and  get  it  and  operate  on 
it  and  get  a  return  of  Y.  A  invested  50  dollars  in  it,  and  B  picks  it  up 
and  adds  something  to  it  and  he  gets  60  dollars  for  it,  so  10  dollars  was 
made.  Now  let's  just  suppose  that  B  didn't  know  that  A  made  it.  A  did 
make  it  for  B's  use  but  B  didn't  know  it.-  What's  the  net  return  on  that? 
Well,  we  spent  5 0  dollars  and  we  get  nothing  back.  In  the  other  case  we 
spent  50  dollars  and  we  got  60  back.  What  was  the  information  worth?  Namely 
the  information  that  A  had  made  it,  for  B's  purposes.  Well,  somewhere  up 
to  60  dollars  -  very  interesting. 

I've  tried  to  say  some  things  which  would  stimulate  some  discussion  but 
maybe  I  have  not-  in  which  case  the  meeting  will  be  that  much  shorter.  And 
if  I  have,  perhaps  now  would  be  the  time  to  try  to  take  up  any  questions  you 
may  have.# 


#The  discussion  period  is  not  recorded  here  since  the  questions  from 
the  floor  did  not  come  through  on  the  tape  recording. 
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